A collaborative comic by Team Train Frontier.
If you're a beginning or intermediate game developer, you might not have worked with or even heard of the term "hotloading". Simply put, hotloading allows you to dynamically change data and have it appear immediately in-game. Good hotloaders work automatically in the background -- just like magic.
XNA was built for a lone developer working on small projects, typically 2D. The Content Project makes it easy for one person to compile code and content at the same time, but requires an extra exporting step for asset and data, with good reasons.
On the other hand, Team Train Frontier is a duo, working in different cities. Working on a 3D game with hundreds of different asset and data files needs to be done asynchronously and quickly, with little upkeep -- and even for a lone developer the time savings could be tremendous.
Hotloading Will Improve Your Life
I suspect the more advanced and larger XNA projects already use content hotloading extensively (e.g. Xen has a particle hotloader). However, there's a lack of detailed information on XNA hotloading on the web, and many of the gems are relatively old.
For Team Train Frontier, coding a data hotloader early on was the best decision made on the project. It took me quite a bit of effort to work out the XNA-centric details for myself, so maybe this post can help guide newer XNA developers.
Unfortunately, there's no hotloading on the Xbox due to XNA restrictions. Don't let that deter you, though, since most of our dev work takes place on PC.
Here's one way to structure your XNA hotloading pipeline:
- Detect changes to data files (shaders,textures,models,xml, etc.)
- Build assets to .xnb files (XNA's own binary format)
- Detect changes to xnb files associated with in game content
- Load in new xnb content
- Apply content changes in game
How our game does it.
Looks easy, huh?
XNB Content Builder Tool
The 1st and 2nd steps were solved by making an XNB content builder tool, that runs separately from the game.
The starting point was the well known Microsoft Winforms Content Loader sample, which uses build projects to call the content pipeline. Due to the lack of updates and complexity, I recommend avoiding XNADevRu.
You'll also want to look at Shawn's Content Pipeline assemblies post first -- easy to overlook if you're new to XNA, due the post's age.
Depending on your project, a separate builder isn't required. You could technically integrate this tool into your game code, and Shawn also has a write up with code on rebuilding effects using alternate ways. If you're willing to forgo all the loader niceties that XNA gives you, you could load all your assets with separate libraries and code and skip XNB building altogether.
One plus to having a separate builder is that it can run without the game open -- allowing programmers to skip between asset building and coding uninterrupted.
My XNB Content Builder simply just asks for the data folder and game folder, then waits silently in the background for new or changed files in the data folder, which will then be packaged up and sent to the game/content/ folder. It doesn't display anything, except for a red or green window indicating if it's active.
This little guy is our XNB Content Rebuilder. That's it.
What goes on behind the scenes is a bit more complex. When it's started up, it checks for new files, which are added to a special Assets XML file (more about this later), and all data files are checked against the content folder files.
If the data is newer than the xnb/content file, that means the content is out of date and the new data needs to be rebuilt and sent to the content folder. When the XNB Content Builder finishes building newer assets, it enters a sleepy state, where it only checks for new changes and files in the data folder once in awhile.
The Winforms Content Loader sample will show you how to use the Microsoft.Build.Execution.BuildManager to build XNB files into your game's content folder.
If you have custom content types (our game has quite a few custom types using XNA's serializer), and you wish to evaluate them in your content builder, then you also need to add your custom type project to your content builder solution and have it referenced correctly by your builder project.
I found it useful to name custom types XML files with a pre/postfix notation, so type can be determined without reading the contents. For example, our material definitions are prefixed with "mat_".
You need to add each of the assembly names of your custom type project and the XNA content type assembly path to the Microsoft.Build.Evaluation.Project using Project.AddItem("Reference", AssemblyName). Again, the Winforms Content Loader will show you how it's done. In my project, the AseemblyName points to the file path of my custom type DLL.
Make sure your set your Project.SetProperty("XnaProfile", ProfileName); to the appropriate Reach or HiDef profile.
Game Data Reloading
Steps 3 and 4 get solved within the game itself. When the content is loaded, each file's last write time is saved, so we can check for newer writes later.
When your game is open, the game needs to keep watch on files, checking every second or so for newer write times. It doesn't need to do this asynchronously. Just keep file watching code to a minimum and out of your Xbox (or even release) builds. My watcher code is a compound loop isolated in an #if WINDOWS block, with a couple dozen statements.
If you're working on a slow laptop like I am, then a good hack is to have the XNB Content Builder touch the folder's last write time, and have your game simply check the folder for new updates (using Directory.GetLastWriteTime()/SetLastWriteTime()). You can't have the game/content folder open in another application using the folder write time hack, but you'll rarely ever touch the game/content folder.
Remember the Asset XML mentioned above? It's not actually necessary, but is useful if you'd like to batch up content to be loaded together. My game code contains an asset manager that loads in a default asset.xml list, then loads all the assets contained in it (including other asset lists). At the moment, Train Frontier in-game content gets loaded all together, but it will be separated in the future to reduce load times. This is also useful if you plan on dynamically streaming in content on Xbox.
You'll also want to use a unique instance of ContentManager for each of your asset groups that you want to load and unload together.
Propagating and Updating Game Content
The last step is tricky, if you're not experienced with building game engines. How will your asset will become visible after getting reloaded? How will your data get updated?
Newer engine programmers will just access loaded data directly, causing all sorts of headaches when attempting to propagate changes to the many entangled parts of code using the data.
However, If you've been a good engine programmer, your content is probably separated by a shared asset system, which will then be able to swap out old content with newer versions, without touching other parts of the engine. Some sort of double reference system is ideal here: have a reference to your loaded content, then give out only a reference-to-a-reference-to-your-content to other systems in your game.
Train Frontier Express has a modular asset system, so each type of asset is handled independently by a type manager.
Tweak a game asset in seconds.
It doesn't take a brain surgeon to see why this is awesome.
All An Artist Needs
Interestingly enough, using this system means an artist only needs a few folders to allow him or her to add new content and iterate on art. No code!
Here's all we need to run Train Frontier Express on an artists' dev machine.
Game folder - contains the game executable, associated DLLs, and the default Game/Content/ folder.
Data folder - contains the game assets, such as XML, textures, models, and other editable game data.
Tools folder - contains the XNB Content Builder executable and any other custom dev tools
XNA Game Studio still must be installed on an artist's machine, though, since the runtime and content pipe are required.
What About Xbox?
Well, you might be asking by now, "This is all fine on PC, but isn't it a lot of work to get a version of the game on Xbox?" Really, it's the same as the traditional way of updating a game's content. Simply add the data to the Xbox 360 Content Project, build, deploy, and run. Visual studio won't deploy files not added to the solution, unfortunately -- probably because it doesn't bother to check for new files in the content folder.
You don't get the advantages of hotloading on Xbox, but you don't get any downsides either. If you've kept your hotloading code clean and separated, your game will run on Xbox without any fuss.
Final Thoughts
Remember, making your assets content hotloadble will not only make your artists' life much easier, but also make your own life easier. Plan early for it, and you'll reap even more time savings.
Imagine reducing interdependence, syncing, and technical support for artists during their daily work! It's saved me time creating and tweaking technical art and shaders. I can change a model's color, textures, shader, name, and even the shader's code, and have it show up immediately in my game window. Pretty cool, I'd say.
I'll eventually get around to open sourcing the hotloader code, though finishing Train Frontier takes priority. Add us on your favorite social network for more updates.
If you've got the chops, then you can start right now on your own hotloader with little problems. Good luck and happy iterating!
-Eric
Links:
Winforms Content Loader
Shawn's Effect and Content Pipeline automation post
Microsoft.Xna.Framework.Content.Pipeline, for using pipeline calls directly
Shawn's Content Pipeline explanation
Empyrean's File System Watcher and Content Rebuilding post
No comments:
Post a Comment