Tip 31 - What is lag?

URL: http://half-life.morat.net/tfc/guides/

Well - it's about time I buckled down and talked about the netcode. Hang on to your seats, this will get technical in parts. Hopefully it will also help you understand why your connection is bad, and how to fix it.

First of all, some definitions:

Due to the confusion between latency, ping and such, here are some definitions I will use.

Tick. Quake engine games wok on the concept of 'ticks' which happen regularly, and everything that happens during a tick is deemed to all happen at the same time at the end of that tick. A tick is, by default (and almost always) 1/10th of a second.
Frame. NOT a network frame, this is being used solely for the in game frames - the action that happens during that tick.
Packet. A network frame, a blodge of data. Usually quite a small blodge. Packets are effectively envelopes containing the game data, which are sent down the wires to and from your computer. Think of them as physical objects which get moved around, and things will seem simpler.
Latency / in game ping. The number that shows up on the scoreboard. The time it takes for the server to process a frame, send the data to your client, thn have your client deal with it, and send back a reply.
Distance. The time it take a packet to move from point A to poiunt B (eg your computer to the server) NOT the round trip time. It SHOULD be the same in both directions, but isn't always.
Lag. Perceptible glitches in your clients mimickry of server state.
Ping / out of game ping. Round trip ime for a packet, that does NOT include processing at either end. Will always be smaller than latency. Beware, gamespy LIES and guess at latency based upon ping.
Framerate. The number of frames per second your graphics card is rendering. Post TF1.5 graphics framerate has little to do with network framerate, unless it drops below 12.
Pipe. A network connection. The bigger or fatter the pipe, the more packets you can get down it at once. This is a plumbing metaphor, and says that packets are like droplets of water.

Now some targets for your in game latency:

If you are connected to the net via a 56k hardware modem, then you should be able to get your ping down to the 90-120 range. 250 is the highest you should ever see.
If you have a software modem, then stop having a software modem as soon as you can.
If you have ISDN then your target ping is about 60 - peaking at 150.
If you have cable, then you should aim just a bit lower than that - 50 to 130 sounds about right.
If you are the lucky b****rd who is still at college and on a LAN conneected direct to a national backbone, then aim for 30-80.

Next, some useful tools:
First of all, your in game displayed latency. There are a couple of things to be careful with when reading it off the scoreboard - mainly that packetloss is munged into the number, which makes things go a bit wonky.
Instead I suggestusing the net_graph command.
net_graph 2 gives all the information you need in text form - net_graph 1 gives it in graphical form, but that does nasty things to your framerate.

Second tool of choice is the dos command 'ping'

And third tool, of choice is the dos command 'tracert'

How to read these tools.

Text netgraph gives you the most in game information in one place. It shows framerate, latency, packet loss, packet choke, uprate (out) AND downrate (in)

Graphical netgraph gives you extra information, it shows you exactly what happened to each packet in turn.
Read it as vertical lines scrolling from right to left. The shorter the line, the sooner the packet arrived - any colour other than green means that the packet didn't arrive.
Red lines mean that the gamehas no idea what happened to it. Yellow lines mean that the server decided not to send it, orange lines mean that your client never told the server to send it, and blue lines mean that it was sent, but got messed up somehow.

Ping simply gives you the ping, but it tries multiple times (defaults to 4) so that you can see whether it is stable or not. Try about 20, and you can see if all the pings are getting through, and whether or not they are stable.

Tracert (pronounced 'trace route') lets you find out WHERE the packets are getting lost.
Effectively, it tries three pings to the first machine after you. Then three to the next one after that, and so on, until it pings the machine you told it to trace to.
The numbers should slowly and steadily increase, but often you will see a big spike somewhere - then you know that machine is the problem.
If you see '*' then the packet was lost. The later onj in the trace you see the *'s the less mportant it is - but a machine that adds a big jump to your ping will also start adding those - and that's bad.

How to interpret these readings. From best case down.

Ok. Stable short green lines, all the tracert is as expected, but the game is very laggy anyway.
first of all, it could well be your graphics settings in this case, check your framerate - and if it's low then try lowering the detail and resolution.
Secondly - it could be that the people you are firing AT are lagged - which with the new netcode can make them jump around on your screen. Note, lagged players do NOT cause lag on the server, this is a MYTH. It could also be that the sever is having trouble - perhaps it's hosting one too many games (ie, more than one)

Game play is fine, but occasionally everything locks up, or goes very jerky.
*probably* this means that you've left your virus scanner, or ICQ, or your email client running, and it's just decided to start doing something. Heck, I've even had scandisk cut in while playing a long map :)
Turn everything off before entering the game.
Second posisbility is that there is currently a bug, when some people enter or leave the server, something about their networking configurations causes the server to do real bad things for a while.
It affects everyone on the server equally, and goes away after a few seconds.
Note that this will also happen if the server is running two games (BAD IDEA) and someone in the OTHER game joins or leaves...
Last possibility is transient network instability. Which is a posh term for 'shit happens'

I can play fine on some servers, but on others, my play is rubbish.
To quote a famous doctor:
It hurts when I do this!
So don't do that.

Servers highly distant from you (note that they might be physically close) will usually give you more trouble than those not so distant. Also some servers are just badly set up, or running off a low bandwidth connection. Finally, your ISP may just have an "I'm not speaking to you" type of argument with that ISP, so your packes are having to go the long way round, or filter through a tiny pipe.

Ok. Say that you HAVE to play upon a 'bad' server - because you know a bunch of the people there.

First of all - are you being choked due to network problems? In that case the way to get a good connection for you is an agressive policy (NOTE - this is VERY impolite to other net users, and I *really* recommend it only for not distant servers. Wait for powerplay, or change ISP is your 'best' solution)

If the problem is with the server itself, then setting a very LOW rate can get you 'some' connection, even though it won't be good. Getting under the servers rate cap is important, and you may even wish to turn down your cmdrate.

Next problem - constant high ping, but no loss (or choke)
Well, probably, it's your modem. Next place to look is your ISP.
Some good generic modem tweaks at Tweak ">http://www.tweak3d.net">Tweak 3d {though ignore their halflife tips} and there are a couple of programs that may be worth getting - mainly 'ispeed' which will tweak a load of registry settings for you.
Other things that can improve your experience here include upgrading your network stack - which you do by upgrading to ME or 2k, or by downloading it from microsoft.com
Get the security patches whilst you are at it.

Next problem. Continal low choke.
(Yellow lines on the graph, regularly spaced out)
This simply means that your rate is not high enough. Turn your rate up, but not higher than your modem can handle, nor higher than the rate cap of the server.

Next - continual low loss.
EXTREMELY low loss is expected and you can't do anything about it. But more than one packet every few seconds means that your rate setting is too high. Lower it a bit, so that your modem/ISP can cope.
Yes, this is a balancing act between too high and too low a rate :)

Next up, corrupted packets continually.
This means one of two things - PROBABLY a dodgy cable. Check that the cables are firmlay stuck in the wall and your modem, and that the cat hasn't chewed through the wires. Replace the cables.
Secondly, it could mean that packets are getting fragmented, and not passed back up to you properly. Turning your maxMTU setting down fixes this - but shouldn't be needed unless you previously turned it up.

Lastly - utterly orrible connection.

Tracert tracert tracert.
Find out WHICH computer is causing the problem.

If it's a *big* router like links, or alter.net, then you are screwed. Your ISP needs a better connection to the net, and you should pester them until they get it, or change ISP.

If it's a router very close to your machine, then it's internal to your ISP. Again, bitch and moan until they fix it, or move ISP.

If it's close to or at the games server, then it is them, or their ISP. Bitch at the games server, or play elsewhere.

If ALL the games servers not distant from you are like this, then emmigrate.

If the problems start from your machine and get worse, then you've got something dodgy in your drivers, most likely. Updating your ethernet/modem drivers is a good idea, as is a full reinstall.

Right, that's the quickie guide over.
Now for an explanation/glossary.

Everything starts with the server. The server is the final arbiter of everything.
The server also holds some history (by default 500ms of it, but thats adjustable)
The server looks at what it knows about you, and sees that there is something about this frame (and maybe past frames if your connection was slow recently) that it doesn't think you know about. It bundles this up into packets, and sends them at you, all nicely stamped and numbered.
[Addition - pre TF1.5 this packetisation was horribly innefficient. EVERY packet would basically contain EVERYTHING about the game state. As of 1.5, things that haven't changed aren't sent. So for example the skin a player has on isn't sent very often. This can lead to weird effects when the packet that contains the default goes missing - it can take a few seconds for the server to realise you don't know what it is, and send you an update. This is the cause of the current skin bug. Who knows what caused the old one? :)]
These packets go down the wire, where, this being a conveniant example, different things happen to them.

Alien mind control rays interfere with one, and mess it up completely, but not enough that it gets lost. It still gets through, but its data is garbage.
A second packet decides to travel by the scenic route, and takes a lot longer than normal to arrive.
The third packet gets stomped on by angry postal workers at an overly busy node, and discarded.
Maybe the fourth and fifth go through normally, but the server decides not to send the sixth - because it's got limited bandwidth, and has to talk to other players too.

Now we wait a while. The packets whizz across the globe through lovely digital communications. Then they hit your ISP. And find that they have to go to a modem. Yuck.
This slows them down considerably, but they make it.

They filter through the operating system, which doesn't crash or anything, but sends them on to the halflife engine.

Halflife puts them on a pile and forgets about them like all the others.

Some short time later, halflife decides to give some time to its networking engine - and has a look at these packets.

The first packet it pretty quickly realises is corrupt, and throws away. Blue line for the netgraph. (No, don't ask what happens if it FAILS to notice corruption - it's undefined. You might be able to get on with the game with only a temporary glitch, more likely you'll end up getting disconnected)

The second packet hasn't arrived yet, but the game does notice that it's missing (the packets are numbered, rememeber?) and puts a red line on the graph.

Same with the third packet - although this one will never arrive.

Fourth and fifth arrive, and the game deals with them. This might involve updating an enemy position, maybe triggerring off a whole bunch of effects (all of a shotgun firing is now a single packet, whereas it used ot be a whole load of seperate ones for the sound, the pellets etc.)
Some of these effects may not be due yet (multiple animation frames) but they are all queued up - along with all the immediate effects. There's some kind of internal stack here. Don't quote me, but I *think* this is what the cl_flushentity problem is. In regard to fixing this, some people have reported that playing around with hardware to sort out IRQs and such helps - so presumeably it's a timing problem. At a guess, this internal stack is growing too quickly from incoming packets, but the bit that gets rid of them isn't getting enough time.
Turning down your rate, or connecting to servers more distant also help, but bring their own problems.

This stack is, at some point, serviced by the graphics engine, which draws players on the screen, bungs rockets on etc.
Also note that it will interpolate positions, doing prediction on player motion until it gets new information, and move rockets and such about on it's own. These are again pushed onto the internal stack.

Ok. Now, the player sees some pretty effects on screen, and pushes some buttons.
This turns into some commands to send to the server. Because it's fun, lets prime a grenade and fire the shotgun at the same time.
Now, depending upon the cl_uprate and cl_cmdrate they might hang around on an outgoing stack for a (very) short while before getting sent out.
Also bunged into packets is info for the server about what packets have arrived, and which it thinks are missing. (1, 2, 3 and 6)
These packets go onto the wire. And hopefully don't get lost. But because we're evil, lets say one does.

Ok, meanwhile, the server has got a new load of data from the players a while ago, updated its internal state, and sends out a bunch more packets. Lets call them 7,8,9 and fred.

Then the server recieves the stuff the client sent it. Says 'ok' and deals with it.
First the shotgun shot. This shot contains both WHERE the shot was fired, and WHEN.
Lag prediction happens on shotgun blasts, so the server looks at it's history to check if you hit anything.
[presumeably, the shot also hangs around in the history, in case a client belatedly tells the server it was there]
The server makes up some appropriate 'you take damage' packets, and puts them on the wire.
Next the grenade packet. Apparently, no lag prediction happens here, so the server notes it down, notes down that it's gonna explode in 3 seconds time, and put the 'sure you primed it' down the wire.
Now, the big nasty packet that says that some packets never arrived. The server deliberately didn't send 6, so it won't resend that. (6 should have been a low priority packet, something unimportant)
However, it does resend the others that went missing. This once again means that the max rate has been hit, so it throws away the least important packet (sorry fred)
It puts all this on the wire.

The client shows blood effects for the target, thinking it hit - this is all done client side, put on the internal stack.
It now receives packet 2, which was merely on a long trip. It takes the red packet off the netgraph, replacing it with a big green one - and deals with it. Belatedly unveiling a spy, perhaps. (So even though it's been firing a while, the disguise only come off now)
7, 8 and 9 arrive - get dealt with as before.
Again the client makes new packets, sends them out. We'll ignore them.
The resent packets 2, 3 and 5 arrive. The client has already seen 2, so it throws it away. The others it deals with. However, it will tell the server that 2 got sent twice - and will now assume that it is pinging a bit higher than it was before - so that it will wait a bit longer before getting impatient and saying packets aren't arriving.


Yeesh.. this is getting long...

Last up - some of the networking commands halflife provides the server and the client.

sv_unlag, sv_maxunlag - turns server history on/off and sets how long it should be.
cl_lc 0; cl_lw 0; cl_lb 0 - turns off the client side lag compensation, blood effects and such won't be drawn until the server confirms them. Turn this off only if on a low latency connection, or you prefer reality to comprehensible fake.
sv_maxrate - the maximum rate setting the server will honour. Usually 5000, sometimes as high as 8000. Not seen it lower than 3000.
sv_minrate - the lowest rate setting the server will take notice of. Usually 3000.
rate - the maximum amount of data your client want's to be sent every second. Betwen 3500 and 5000 is a good number for modem, between 5000 and 8000 is good for cable.
cl_cmdrate - the number of times a second the client tries to tell the server what it is doing. 30 should be fine, don't lower it further than 15, but turning it up can make things go a bit smoother on fast connections.
cl_uprate - the max amount of data your client tries to send the server per second. You won't need to turn this up unless you're uploading decals. 3000 is a good setting for modems. If you've got cable or better and hav turned up cmdrate you'll want to turn this up too.
cl_allowdownload and cl_allowupload - allow upload and download of maps and decals. You will want to turn these off, for speed.
cl_resend - the number of times to try and resend a packet. Set to either 1 or 2 - since more traffic just uses up rate, and old data is probably out of date and useless anyway. But expect this to lead to more errors like people with the wrong skin.
cl_updaterate - the client asks the server to try and send this number of updates per second. Setting it higher than your framerate is a total waste of time. Set it lower to have a more stable game, but expect more innacuracies as lag compensation is not perfect. 20 is fine.
delta_clear - flushes out some internal stuff. DOn't play with it, but might help you recover from a real bad lag spike.
fakelag , fakeloss - the first takes numbers as *ping* _NOT_ latency. They fake that your connection is worse than it is, so that you can practice playing under lag on a LAN, or make games fairer on other people. Note, use a small amount of loss, then set lag to half the difference between your latnecy and your dessired latency. A small bit of loss is worse than a lot of lag.
fps_max - your max graphical framerate. Unless stupidly high, should have little effect on your networking. Setting it low gives you a LOW but STABLE framerate, and this feels less laggy, and is easier to play with than having a high framerate that jumps about.
host_framerate - server side, lets you play with the tick speed. Don't.
host_limitlocal - I *think* this limits cl_updaterate and cl_uprate. Server side anyway.
host_profile - who knows :)
r_mmx - not a netowring command, but can screw you up completel. Try turning it off.
set system ticrate - play with the tick rate. Don't.


And finally:
Agresisve strategy.
Set your rate as high as the game will let it go. Send and recieve twice as much information as you need. Only half of it will get through, but that's enough to get you a good game.
Please note, this is distinctly unfriendly to other players.

Minimal strategy.
Turn your framerate and such WAY down - and lower your rate. You won't get as muh information, but since you're sending so little it should get priority, and fit into the netwrk space allocated to you.

This only works if the routers your ISP uses are smart, and are doing load balancing of some type.

Whew. The end.
Disclaimer, I don't work for valve - this information is pieced together from playing the damn game and interviews they publicly gave (mostly yahn)
If it's wrong, I don't care. But tell me, I may fix it.
If it helps you, I'm glad.
If it blows away your computer, it saves me a rocket. If you whine about it, then you obviously needed to learn more about computrs before acting, and I can probably glean ICBM coordinates from your whine.
If your cat chews through your modem wires, it's probably trying to tell you something. Listen to your cat.

i