So I heard Ping based Unlagged is going

killall -9

Moderator: Developers

Post Reply
User avatar
Dark-Assassin
Maintenence Crew
Posts: 962
Joined: Fri May 25, 2012 4:25 am

So I heard Ping based Unlagged is going

#1

Post by Dark-Assassin » Sat Apr 30, 2016 5:59 am

Apparently to be removed from 3.0

Just wondering if there is any major reason for this?
Only asking because with a recent game I've had (haven't played properly in a long time), which was an Instagib game, I've noticed a few differences between tic based unlagged and ping based unlagged for me. My ping was roughly 250-300 (which is normal for me) on the server.
When I was using the default tic based unlagged, I found that I still had to aim ahead of my target to hit them, but only a little. It's uncomfortable and throws off precise aim. And that was regardless of cl_ticsperupdate set at 1-3
However with the ping based unlagged, it was almost perfect for me and my shots were making contact where they should, which is the player and not in front of their moving direction.

I guess what I'm asking is there any chance to keep this if enough tests show that high pingers such as myself get better results with ping based unlagged over tic based.

Note: This was only tested in 2.0, yet to confirm of 3.0 is any better

User avatar
Doomkid
Frequent Poster Miles card holder
Posts: 908
Joined: Sun Nov 25, 2012 2:04 am
Location: Aussie Land
Clan: UniDoom
Clan Tag: [UD]
Contact:

Re: So I heard Ping based Unlagged is going

#2

Post by Doomkid » Sat Apr 30, 2016 6:33 am

I'll be sure to compare the two over my next few matches as I often ping around 300. Either way, choice is always good!

User avatar
Torr Samaho
Lead Developer
Posts: 1496
Joined: Fri May 25, 2012 6:03 pm
Location: Germany

Re: So I heard Ping based Unlagged is going

#3

Post by Torr Samaho » Sat Apr 30, 2016 8:04 am

Dark-Assassin wrote:Just wondering if there is any major reason for this?
All reports we got indicated that tic based unlagged is working better than the ping based approach in all cases. Thus, there seemed to be no reason to keep the inferior method around.
Dark-Assassin wrote: I guess what I'm asking is there any chance to keep this if enough tests show that high pingers such as myself get better results with ping based unlagged over tic based.
If there are clear cases where the ping based approach is better, we can surely talk about keeping it.
Doomkid wrote:Either way, choice is always good!
While I agree with this statement in general, you have to keep in mind that an abundance of choices can do more harm than good. Look at all our dmflags for example. We have gotten to the point, where it is difficult for players to agree on a set of settings.

User avatar
doomista
Forum Regular
Posts: 142
Joined: Sat Mar 07, 2015 6:58 pm
Location: I've been to hell. Twice

Re: So I heard Ping based Unlagged is going

#4

Post by doomista » Sat Apr 30, 2016 9:07 am

Torr Samaho wrote: While I agree with this statement in general, you have to keep in mind that an abundance of choices can do more harm than good. Look at all our dmflags for example. We have gotten to the point, where it is difficult for players to agree on a set of settings.
There are certainly more things to be simplified. For an instance, I never understood why Set Video Mode is not called Resolution and why it's not under Display Options (or maybe better Graphic Options). I quite often click the wrong option. And now I found out that the new Zandro threw a lot of categories out of Display Options to main Options menu. I understand there are probably some relics of the (G)Zdoom code since zandro is their multiplayer fork, but is it really necessary to keep even things that are bad or weird? Like "Enable 5:4 aspect ratio". With the default setup on yes I can see no reason why to have it after all.

User avatar
Doomkid
Frequent Poster Miles card holder
Posts: 908
Joined: Sun Nov 25, 2012 2:04 am
Location: Aussie Land
Clan: UniDoom
Clan Tag: [UD]
Contact:

Re: So I heard Ping based Unlagged is going

#5

Post by Doomkid » Sat Apr 30, 2016 9:53 am

Torr Samaho wrote:While I agree with this statement in general, you have to keep in mind that an abundance of choices can do more harm than good. Look at all our dmflags for example. We have gotten to the point, where it is difficult for players to agree on a set of settings.
This is definitely a good point. I will also say that I agree with doomista, but with so many options it does naturally become kind of difficult to sort them all. I've often found myself a little lost in the menus even after all these years, but by the same token I can't really suggest/think of what a better approach would be.

Konar6
IRC Operator
Posts: 455
Joined: Sun May 27, 2012 9:38 am

Re: So I heard Ping based Unlagged is going

#6

Post by Konar6 » Sat Apr 30, 2016 10:45 am

Did you play on the same server as with ping unlagged? Not having played for a long time may also be a factor. You should do your tests on the same server in one sitting, otherwise it might be just placebo.

User avatar
Dark-Assassin
Maintenence Crew
Posts: 962
Joined: Fri May 25, 2012 4:25 am

Re: So I heard Ping based Unlagged is going

#7

Post by Dark-Assassin » Sat Apr 30, 2016 12:12 pm

I've switched between on the same server a few times, and really found the ping unlagged works best for me. Easy to tell with a railgun and players moving everywhere.

Catastrophe
Addicted to Zandronum
Posts: 2558
Joined: Sat Jun 02, 2012 2:44 am

Re: So I heard Ping based Unlagged is going

#8

Post by Catastrophe » Sat Apr 30, 2016 3:40 pm

Torr Samaho wrote: While I agree with this statement in general, you have to keep in mind that an abundance of choices can do more harm than good. Look at all our dmflags for example. We have gotten to the point, where it is difficult for players to agree on a set of settings.
Tbh isn't it better that way? Gives more variety/choice.

Watermelon
Zandrone
Posts: 1244
Joined: Thu Jun 28, 2012 9:07 pm
Location: Rwanda

Re: So I heard Ping based Unlagged is going

#9

Post by Watermelon » Sat Apr 30, 2016 7:05 pm

EDIT: While my suggestions for helping us fix this issue is maily for Dark-Assassin, this can go for anyone else as well:


I'm going to outline a brief explanation of how both work (of which I'm not implying the people in this thread do not know how it works, but more or less for everyone -- including casuals -- who view this and aren't familiar with the mechanics) and then proceed to conclude what our best options to take are from that. There is some interesting research I stumbled across over the past 2 years, so parts of this post will be interesting to everyone from all areas hopefully.

How unlagged works is it keeps track of the past 35 or 70 (or some number of gametics, it's been probably > 1 year since I looked at the source code... lets assume enough for at least 1000ms of lag compensation), and when your fire command arrives on the server: it rolls everyone back to the recorded position, does the attack, and restores everyone to their position. This all happens instantly and you never see it on your client since it's all serverside.
  • Ping: The method of ping based unlagged is to use the time it takes over the net and divide it by the amount of milliseconds per gametic to tell you 'it was this many gametics in the past', of which you then apply the above to get your desired result.
  • Gametic: Gametic unlagged is where you know the (maximum) gametic of the last update, and emit that with your attack so the server knows the last frame you saw when firing and will apply the appropriate rollback to get the result.
The key to understanding what goes wrong with both as we deviate from ideal situations (where ping and gametic would be identical in performance):

With ping based unlagged, the server sends out a pulse to the clients every second, of which the client responds with the timestamp back to the server. From this it can tell how long the trip took, and update the player ping accordingly. This means you only have a resolution of one second, and anything that goes abnormally wrong with packet delays in between will result in inconsistent reconciliation. For example, suppose we fired off our packet to the server which arrives and says we have a ping of 30ms. Now suppose 100ms later we click fire and on the path through some nodes on the net, it gets routed wrong or something weird happens, and it arrives at the server 250ms later. Our shot will then be rolled back 30ms (which is 1 or 2 gametics), but this is far from how many gametics it was when we actually saw the player (approximately 8-10 or so). The way the unlagged code worked was grab the most recent ping, and since the time slice is one second, depending on when the routing goes weird (or worse, the situation I dealt with and many others which will be described in a second), it will still use the "30ms" unlagged because it has no idea about the ping fluctuations that occurred until it requests an update approximately 1 second later.

Where this continues to go south is with how routers are notorious for screwing up and holding onto packets when other things occur on the network... in particular wireless. I noticed a significant difference between wifi and wired, my shots felt like they were consistently hitting wrong while on wireless, and switching over to wired not only fixed this problem [almost], but somehow I was out-reacting people who out-reacted me originally. Maybe it was the distance from the router, I'm not sure, but it brought out all the negatives of ping unlagged. Furthermore, a vast amount of the community does not own their own place and has to deal with others using their internet (which further exacerbates these issues), and then we have to throw in the beyond-our-control-handling of our packets from our ISPs, many things can go wrong with a faulty node.

With gametic unlagged, we see are sending the most recent command we've received from the server. This bypasses all of the problems of ping unlagged (except a small amount of cases which I will describe shortly) because you are only sending the latest update that you received. If you happen to have some major spike with your 'fire' packet and it goes around in some horrid 400ms loop, it contains the time-stamp of exactly what gametic you saw when you fired. From this, when the packet arrives at the host, the server knows when you fired and will adjust accordingly. No kind of vicious spike [within reason] will cause your shot to come out weird because you are effectively transmitting the index of the time-slice that you are -- quite literally -- seeing.

There are some areas where gametic can have problems. These are when packet ordering becomes fuzzy and another packet arrives before our 'fire' packet. Servers can't just let clients pick and choose the gametic to apply the shot, or else hacked binaries will just look back in time for the proper place and send those. Only the most recent gametic received from the client is used. This means if you send two packets: F (for fire), then M (for move, where F comes after by 28ms), if packet F gets lost along the way but somehow M does not get routed badly and arrives first with a more recent gametic, then the stale gametic in F will be ignored and the unlagged will process the most recent gametic (which we're assuming M is at least 1 or more gametics ahead of F since it is emitted from the client one full gametic later... assuming a steady stream of packets). Usually its not the case where just that one single packet is toasted, and usually clients tend to send "+fire" in their packet over multiple, so such cases are rare for this to happen -- but there is one interesting result I discovered online:

There was a casual investigation by a network programmer into the effects of ping when crossing the ocean (where ping levels were 200-300 or higher), and he found that with really high ping, even under stable ping conditions, the ordering the packets has a much higher probability of arriving out of order. Why? We don't know, but this was just a result that was discovered. This means you may be suffering possibly from this issue. However I'm definitely hesitant to say that you are suffering from this and its not some other anomaly, but I'd like to keep this option that your high ping may bring out a fringe corner of gametic unlagged on the table.

So that covers one potential issue, and I'll discuss shortly how we can debug this issue with your help to figure out if this is in fact the issue.

Another problem with Doom is that other games aren't as fast as doom, so such anomalies if being off +/- 1 gametic do not have as critical of change as Doom does. We don't tolerate much error as well as other games since one gametic and a player moving 30 units per second on some axis means being off by a gametic of 1 could be off by the whole size of a doomguy. When we include the railgun and throw out guns like the SSG with really wide spread, it's easy to run into serious issues. Other games do use gametic unlagged and have immense success, and you can see this in games like Half-Life where even at 200ms the game is very playable. Valve has whole articles dedicated to gametic unlagged and it's effectiveness, but Doom definitely blurs the lines due to speed and object size being opposites compared to the mainstream titles.

Now that we know about how unlagged works, how problems occur on the network side, and why accuracy really matters with the railgun and such, is there any other problems that can happen?

The following may also contribute to shots having problems, along with their provided reasons:
  • We did in fact miss and it just feels like we should have hit them (which we will rule out shortly).
  • Our aim was trained to a specific thing over time and we adjusted to weird compensations that are removed when we use something like gametic unlagged (if we were used to ping unlagged). There was a study showing how if we made a monitor change color when we press a button on a keyboard with a built in delay of some amount of time like 200ms, eventually our brains lock into this 200ms time frame, and when the user tries it after a bunch of tries at 50ms, it feels to the user as if the color is appearing before we even press it. This is some kind of placebo effect to try to watch out for. I'm not saying you are having this, but it's something to keep in mind.
  • There is an underlying issue with how things are processed in the engine that causes the anomalies described.
There's only one real way to fix this, and I HIGHLY encourage you to do it because IMO if you are suffering from these problems, they need to be debugged and proven to either be a connection issue, an aim issue, or an engine issue.

1) Record a demo of this effect to confirm this is in fact an issue.
2) Get a custom build where the server prints out the X/Y/Z of you firing, and the X/Y/Z of the enemies and your unlagging amount when it happens... which will require you and a server to have a modified binary, so this is no small feat of work like (1).
3) If there appears to be an underlying issue, then developers will investigate it.

I know this is a bit of a pain in the ass, but its the only way to find out. From (2), we will be able to determine if you are in fact aiming at the right spot and you are getting unlagged improperly, or if there is a packet ordering issue (though ping based unlagged would also give you these issues but it's worth investigating IMO since it then we can decide if we want to add more data to mitigate this problem), or diagnose if there's some kind of error in the engine itself... which could even go as deep as the internals of the engine that almost none of us touch. Who knows.

As a final story, I find anomalies still happen and things aren't perfect. Gametic unlagged has tremendously helped my shots connect better, and while I've heard overwhelmingly positive responses (which Torr has also seen) and no negative responses, with the fast paced online doom gameplay and problems with collisions of hyper-speeding small bodies, netcode will always be an issue unless we migrate to something very hackable (which then precludes itself from getting in for this very reason). If something was broken, a lot more people would likely have complained by now -- especially in the competitive scene -- that their shots we're getting toasted significantly more on gametic than ping unlagged. However, almost no one plays with 200-300ms, and virtually 99% of the competitive scene that cared about this upgrade are all in the range of 10-170ms of lag, so such corner cases with upwards of 300ms of ping likely didn't get tested for such critical aim issues. I did test this at obnoxious levels like 500ms+ and had no issues, but actually having a client connect across an ocean is a different thing that I couldn't directly test. I still think however doing the above suggested methods will help us discover more about any problems about the engine and deliver a better experience. I'd also like to note that videos on their own will only show that you did shoot at someone, their rail went through their body, and nothing happened. This needs to be shown -- or better yet, showing that you can miss and still kill is the interesting part.

While I'm not around much since my internship has me programming from 8 - 5 and I come home usually wanting to do something else other than touch a computer, I might be able to investigate this on weekends and see whats up. I just need the raw data and videos of someone slowing it down so we can see it happening, and go from there. Even if I'm not around, I'm almost positive the other devs would be able to make a much more educated decision with data.

In short, I would consider ping unlagged being better than gametic unlagged to be a bug that needs to be fixed, not a reason to keep ping around. I am all for keeping ping around until gametic is fixed -- assuming there is actually something wrong with it and not some of the other non-bug related issues stated previously -- though we need to collect evidence for this, and likely by collecting evidence we will then be able to fix such issues.

User avatar
Dark-Assassin
Maintenence Crew
Posts: 962
Joined: Fri May 25, 2012 4:25 am

Re: So I heard Ping based Unlagged is going

#10

Post by Dark-Assassin » Sun May 01, 2016 4:21 am

I'll see what I can do about getting demos and videos.

About the custom build, that's not something I'll be able to make myself, would need help with that.


Although, I'm thinking even a 1 or 2 +/- tic adjustment cvar could work, would have to be capped on the server side too to prevent abuse.

User avatar
WaTaKiD
Master Server Admin
Posts: 88
Joined: Mon Oct 08, 2012 4:40 am
Location: USA

Re: So I heard Ping based Unlagged is going

#11

Post by WaTaKiD » Sun May 01, 2016 6:14 am

i can help out with making builds, albeit for windows only

User avatar
StrikerMan780
Forum Regular
Posts: 269
Joined: Tue May 29, 2012 9:16 pm
Clan: Shadow Mavericks
Clan Tag: [SM]

Re: So I heard Ping based Unlagged is going

#12

Post by StrikerMan780 » Sat Jun 18, 2016 3:14 pm

Strange that you'd get the complete opposite of what many, many (and rather thorough) tests have shown. In our instagib tests, ping unlagged caused quite a few missed shots when they very obviously should have been hits, where gametic unlagged pretty much registered perfectly every time. Quite a bit of noticeable inconsistency in ping unlagged using the unlagged debug actors as well.

User avatar
Dark-Assassin
Maintenence Crew
Posts: 962
Joined: Fri May 25, 2012 4:25 am

Re: So I heard Ping based Unlagged is going

#13

Post by Dark-Assassin » Sun Jun 19, 2016 4:15 am

Sorry that I haven't been testing the issue any further.

@WaTaKiD, could you provide me with a build? However I'll also need a linux server binary too.

User avatar
WaTaKiD
Master Server Admin
Posts: 88
Joined: Mon Oct 08, 2012 4:40 am
Location: USA

Re: So I heard Ping based Unlagged is going

#14

Post by WaTaKiD » Sun Jun 19, 2016 6:22 am

Dark-Assassin wrote:@WaTaKiD, could you provide me with a build? However I'll also need a linux server binary too.
first i need to be supplied with the code in which to build, its quite beyond my capacity to actually do any kind of coding, all i can do is build code others have written

as for linux, im sure someone else could provide a linux build, again provided we get some kind of code to build

Edward-san
Developer
Posts: 378
Joined: Fri May 25, 2012 8:14 pm

Re: So I heard Ping based Unlagged is going

#15

Post by Edward-san » Sun Jun 19, 2016 7:23 am

Do you need a build containing only the tic based unlagged? If yes, latest 3.0 beta should suffice.

User avatar
Dark-Assassin
Maintenence Crew
Posts: 962
Joined: Fri May 25, 2012 4:25 am

Re: So I heard Ping based Unlagged is going

#16

Post by Dark-Assassin » Sun Jun 19, 2016 11:59 am

Edward-san wrote:Do you need a build containing only the tic based unlagged? If yes, latest 3.0 beta should suffice.
Watermelon wrote:1) Record a demo of this effect to confirm this is in fact an issue.
2) Get a custom build where the server prints out the X/Y/Z of you firing, and the X/Y/Z of the enemies and your unlagging amount when it happens... which will require you and a server to have a modified binary, so this is no small feat of work like (1).
3) If there appears to be an underlying issue, then developers will investigate it.
Unless something has happened with the tic based unlagged between v2 and v3, just a modified v2 binary should do.

User avatar
Torr Samaho
Lead Developer
Posts: 1496
Joined: Fri May 25, 2012 6:03 pm
Location: Germany

Re: So I heard Ping based Unlagged is going

#17

Post by Torr Samaho » Sun Jun 19, 2016 5:00 pm

Dark-Assassin wrote:
Watermelon wrote:1) Record a demo of this effect to confirm this is in fact an issue.
2) Get a custom build where the server prints out the X/Y/Z of you firing, and the X/Y/Z of the enemies and your unlagging amount when it happens... which will require you and a server to have a modified binary, so this is no small feat of work like (1).
3) If there appears to be an underlying issue, then developers will investigate it.
Unless something has happened with the tic based unlagged between v2 and v3, just a modified v2 binary should do.
No need to start with a custom binary. First make a client side demo with 2.1.2 that shows that ping based unlagged is more correct than tic based unlagged for your connection. Once we have that, we can use a custom binary to get further information on what is going on.

User avatar
Dark-Assassin
Maintenence Crew
Posts: 962
Joined: Fri May 25, 2012 4:25 am

Re: So I heard Ping based Unlagged is going

#18

Post by Dark-Assassin » Sun Jun 19, 2016 5:09 pm

I'll dig around my demos tomorrow and hopefully find some instagib matches where it's quite clear. Else, a testing session with lots of variables will help too.

Post Reply