The seperation of content and engine

Well, there's no point pretending to be unbiased. This column has two purposes. First, to educate you as to the issues - and secondly to persuade you that seperating content from engine is a good thing. So there's the conclusion. You should strive to keep content seperate from engine. The end.

It should be noted that all of this is from a games perspective - mostly from the point of view of someone who creates, but I'll not totally forget the consumers.

Now move on to the argument...

What Am I Talking About?

First of all - content is 'stuff'. Stuff that you can see, stuff that you can hear, stuff that you can think. Such things as 'plot' are content - although it is fairly hard to define exactly where 'plot' leaves off and 'story' begins.

This text that you are reading here - that is content. The sidebars, they are less so; your web browser that you are reading this in - that isn't.

Engine is the thing that delivers this content to you, and (hopefully) lets you interact with it. A web browser, your printer, the physical book - that is engine.

I'll also define CDL here. A CDL or Content Description Language is somewhere in between the two. It is the glue that binds content and engine together. For example - the english text you are reading has been written in HTML. Heck, the thoughts you are thinking as you read this have been written in English. Both of those are CDL's.

Historical Perspective

The earliest storytelling tradition is the oral tradition. People would tell each other things. The only 'engine' involved here is people - their voices and their memory. The content moves from one to the other unencumbered.

Moving up a notch, from cave paintings right up until the renaissance content was transferred mostly by voice still - but you also had paintings and writings. Copied out by hand, you now had the difficulties of grammer and vocabulary. Unlike a face to face meeting, when writing a book you can't stop, go back and explain if your audience happens to not understand a word. So in exchange for convenience and permanence you lose some interactivity. But engine and content are still inextricably entwined. The form is the message.

Then came gutenberg. The printing press gave another advantage to the written word - mass audience. Suddenly you had more conventions thrust upon you - you had to choose a font. You had standardised way of emphasising words - and mass readership slowly forced the use of standardised spellings.

Theatre has existed for a long time before the printing press. A theatre troupe would (effectively) read a book, and turn it into a performance. In a way, the theatre could be seen as an engine - although they add their own content to create something more unique. (And can create their own plays without ever releasing the script). Cinema changed that.

Once again, with film (including TV) you had (practically) perfect replication. The machinery of cinema is an engine for reproducing and displaying the same content (the ONE performance of the actors that makes it to film) over and over.

Now, let's move in to games. Games bring interactivity into the equation - but also the concept of incremental improvement. With only a few notable exceptions, a game released four years ago will not sell today. The same just isn't true of anything else. A book is still readable fifty or a hundred years later. A film is usually watchable a decade later. Things do change in those media - but nowhere nearly as quickly.

Games are, of course, my chosen topic.

Games

A game tends to have many seperate pieces of content. Graphics, sounds, music, plot - even gameplay can all be considered seperate pieces of content. But they are all wrapped up in a single package, given an engine, and shipped out.

Older games tended to be inextricably linked. Elite on the BBC for example got the (now famous) planet names directly from the program code. Newer games tend to have explicitly seperate content and engine - for some parts of the content. This is being done for several reasons, most to do with it just being easier that way.

The important part (and the part I emboldened) is that not all of the content is held in nice extension files. The intangible 'this is how the game plays' is usually still implicitly bound up in the engine. Most quake-engine games feel like quake games, and so forth.

Games are moving in the direction of positively encouraging modding - it extends the shelf-life of their games and sometimes gives them talented games teams to buy-out and release their next game with. But we are missing one important thing.

You can't run a quake mod in Unreal Tournament

That basically sums up my problem. There are a bunch of different file formats and CDLs used to package up your new content for a game - and they mean that it will only work for that one game. That's fine this year - but who knows which engine will be successful next year?

Some of it isn't a problem. Audio files can be automatically converted from mp3 to ogg-vorbis and back. Texture files can convert from jpeg to tiff. Even model files can usually go via dxf to anything else. (Thank you, dxf) We have tools to convert most things to a 'baseline' standard - a baseline which keeps advancing.

But some things just don't convert well. Levels tend to be too dependant on specific engine features. Even those few tools that do exist to perform conversion (doom->quake, quake->quake3) don't do a very good job of it, simply because the formats are too different.

And some things no one is even thinking about converting, it seems. That 'what my game is like' feel is coded in C, or in a scripting language - and relies totally upon engine features. It can't be converted seamlessly from one version of the engine to the next, let alone to a different engine.

Level Conversion

Annoyingly, maps could quite easily be converted between any application. Any map can be expressed as polygons and some form of control logic. {Ok, not quite any map - you need some standard way of expressing parts of maps that break euclidean geometry}

You can easily convert any map TO this format. Heck, then engine that you play the map with is doing it continuously and sending those polygons to your graphics card to render. The problem is converting back again.

Many map formats have explicit restrictions on what can be done. For example, in quake, you cannot have convex polygons. In lithtech, you cannot have concave 'air' (or empty spaces) - so currently conversion tools are slow and produce innefficient maps. In the future, this will improve - and a little more innefficiency will be tolerated in exchange for faster (and less restrictive) map design. The artists who design the levels in the first place would much prefer not having to know about these kinds of restrictions.

Look and Feel Conversion

Is there any real reason why we can't have a similar system for converting quake mods to UT mods and so forth? After all, one of the main activities of mod teams nowadays seems to be recreating well known game types on various new games.

It seems that every game acquires a TF mode, either before or after it is released. Also some kind of squad tactical 'realistic' shooter. And deathmatch. And co-op. And so on and so on...

What is needed is some kind of common baseline CDL for gametype. Obviously, it will need to be, in some way, a programming language - but it should be divorced from the main game engine itself.

Then, you would be able to simply run a converter and adapt your base code to a new engine, when one comes along.

I don't think that this is an unreasonable request. In fact even without a single unified CDL, mod teams can acheive a similar effect simply by following good re-usable code practices.

Uh, I can give examples - quake1, 2 or halflife. Contact me if you really want them - they'd just make this article even longer if I pasted them in here.

Desire

Well - I've said what I want, but why do I want it? Let's look at this from four different points of view.

Consumer

I want games. I know what game types I like, and I want them to always look as shiny as possible. Witness the sales of graphics cards that add new rendering tricks (such as truform blobbiness, or anti-aliasing) to old games to spruce them up. The ability to quickly port old content to new engines means that I get those old games again and again.

Mod Teams

I can write new games, since I will have more time to do different stuff. Converting all the old popular genres doens't take very long, and I can get on with innovating.

Semi-proffesional Teams

Since only I have the version of my game-mode written in CDL, only I am able to release new versions of it for newer games. And doing so ensures me continued popularity and (possibly) revenue. Furthermore, I can continue to innovate and improve the game on ALL platforms simultaneously, instead of having to throw away an old following in order to satisfy a new one.

Proper Game Companies

As soon as I release my new engine, it will have many games available for it. As soon as I release my new game, it will run on many peoples machines without me having to do as much work supporting all possible types. Either way, I'm in the money! My competitive advantage over other games companies will be my lead time on new features. Only my engines / games will have the absolute newest innovations - by the time my competitors find ways to utilise them, I will be ready to release the next version. Finally, I will never be more than a single generation behind a competitor, since I can use their cast off mods.

Justification

So - will seperating content and engine actually be able to bring about these posited benefits? Obviously, I believe so.

Modularity is generally a way of producing the same thing faster - and allowing you to reproduce it in changed circumstances. The increasing pace of development - both amateur and proffessional demands that this occurs.

As another benefit - standardisation to a single language allows skills to be more quickly transferred. The language itself would be formed by consensus (each game engine adding to it, and the useful parts being carried across to the baseline) and thus would naturally fit the type of applications it is being used to write. Why should "It's a teamfortress game" not be sufficient code to be a basic implementation?

What would this CDL look like?

That, I'm not sure about. It would almost certainly be something based upon xml. I suppose it's too much to hope that we never have to edit the text and instead interact with the structure.

Clearly the language would have to be procedural - C is the obvious choice to base it upon, since so many people are familiar with it. Perhaps a pre-existing c-like language could be used. Python, perhaps, or java.

The support structure would be event driven, and entity-oriented (Entities being like objects, but with a 'real' implementation - programmers can think of them as safe but 'volatile' objects and mappers can think of them as items.) Almost all modern games seem to be being created this way.

But if you carefully seperate out the events with a translation layer, you will have the desired portability. So your code will have an "Event, player fired gun" entry point, that will perhaps be called from "OnWeaponFired" in one game and by "Network - From Team One, Popgun" in another.

Frankly, this is just good coding practice in general - even if you don't have a common language, it will make converting code to future engines much easier.

Fears

I have not addressed the issue of piracy here. If your code is simpler then yes, it will be easier to steal. Perhaps some kind of identifying tag inextricably linked to compiled binaries might be used? Of course - ideally, you would be happy for people to enjoy your code any way they can.