0002859: Gametic-based unlagged seemingly goes out of sync often compared to ping-based unlagged

Notes
(0015790) unknownna (updater) 2016-10-08 04:31 edited on: 2016-10-08 10:39	You can also reproduce this by simply joining the game with the observing client, no need for a changemap map change. I did some more tests and can at the moment conclude with that at any given ping, the red debug actor is further behind the player when using gametic-based unlagged.

(0015907) unknownna (updater) 2016-10-11 20:12 edited on: 2016-10-12 05:39	If this will not be looked into and changed/fixed before 3.0 is released (which I assume is too late now), then I insist that we temporarily reintroduce ping-based unlagged as an option for users. A proper gametic-based unlagged would perhaps be ideal, but in its current state it's simply not good enough to fully replace the tried and true ping-based one. You're going to have to fix the desyncs first before fully replacing it. The fact that it desyncs with a local connection suggests that it's fundamentally bugged. Won't the over-reconciliation technically allow you to hit other players in your own client's past? I did some more tests and can confirm that the gametic-based unlagged also desyncs a little consistently every 24-25 seconds. In addition to this, I now just discovered that it also desyncs whenever you respawn. And not too surprisingly, it also seems to desync after map resets. It also seems to desync with cl_ticsperupdate 2/3, causing the client to seemingly over-reconcile the other players permanently, though I'm not sure whether this is a bug or some visual side-effect. In any case, the debug actor is synced perfectly with the player position when using ping-based unlagged, whereas it's always flickering behind the player position when using gametic-based unlagged. To summarize my findings so far: * Desyncs after changemap map changes. * Desyncs when you join the game. * Desyncs a little consistently every 24-25 seconds. * Desyncs when respawning. * Desyncs after map resets. * Seems to permanently over-reconcile other players with cl_ticsperupdate 2/3. * For any given ping, even with ping spikes, the position of the debug actor is further behind the player compared to its position when using ping-based unlagged. Edit: I noticed that disabling the tic buffer seems to improve things. It then stops desyncing after changemap map changes, map resets and when respawning. Though this makes me wonder if this whole issue is nothing but a visual side-effect of the tic buffer and a misunderstanding on my part. Let me know if this is the case. The ping-based unlagged also seems to undergo a minor readjustment every 25-30 seconds as well, provided that the client has some ping. It doesn't seem to do it with a local connection like the gametic-based one does. However, after some extensive tests with actually firing at a moving client, it seems to me that the hitbox is actually more often than not further ahead with the ping-based unlagged, as the player seems to have a bigger "dead-zone" behind where no pellets hit, and a bigger "hit-zone" at the front where pellets hit. This would mean that the ping-based unlagged is the more inaccurate one? Interesting. So in conclusion, the gametic-based unlagged seems to actually be more accurate, but it still seemingly has some quirks left, particularly with how it starts to "desync" when having a local connection to the server.

(0015915) AlexMax (developer) 2016-10-12 06:17	Here is the original thread where this was discussed. For some reason, all of the links to bitbucket are gone, but hopefully it should point you in the right direction to the source of when the patch made it into Zandronum: 'https://zandronum.com/tracker/view.php?id=1705 [^]' I find your described behavior incredibly perplexing. At least when I was working on the patch, there was no big redesign of unlagged. Instead, it simply tried to measure the difference in tics between the client and the server more accurately, by simply comparing two tic counts instead of relying on ping, which is a much more mercurial and out-of-date value, averaged over several samples. To wit, the original code that picked the gametic to go back to: > int unlaggedGametic = ( gametic - ( player->ulPing * TICRATE / 1000 ) ); compared with what happens now: > int unlaggedGametic = pClient->lLastServerGametic + 1; Looking around in the source, you can see that in sv_main.cpp, lLastServerGametic is read directly off the wire from the client's movement command. In cl_main.cpp, you can see that CLIENT_SetLatestServerGametic() is updated when connecting, and also when the client gets MoveLocalPlayer commands. However, if you check post 8277 on that ticket I linked, you will notice that I did notice a discrepancy at higher pings between the two methods and couldn't explain all of it. Most importantly, I could not figure out if the discrepancy between the two was because gametic was more accurate, or because I was making a bad assumption somewhere. This is where my involvement with the patch ended and Water took over, and it didn't look like anybody followed up on investigating this discrepancy - Water assured the ticket that he had tested it, and I suppose players considered it better than the status quo and wanted it in 2.0. Hopefully some of this rambling is helpful to you. And yeah, I definitely think that Torr should get involved to make sure that our assumptions are all sound.

(0015924) Ru5tK1ng (updater) 2016-10-12 16:27	Recovered the full commit link.'https://bitbucket.org/Torr_Samaho/zandronum-old/commits/8e43651ce72ae4b0a66ebc0e1c011b04c883ac74 [^]'

(0015988) unknownna (updater) 2016-10-14 13:47 edited on: 2016-10-14 13:52	Thanks for the response, AlexMax. Hopefully we can sort this issue out. 0002712 confirms some sort of desync caused by the gametic-unlagged working with the tic buffer. However, even with the tic buffer off it still desyncs a lot if cl_ticsperudate is 2/3.

(0016097) JKist3 (reporter) 2016-10-27 22:18	noob question: how do I change between tic based and ping based unlag? I can't seem to find the variable/command. Also just so I completely understand, what exactly does unlaggedactor.wad do/show?

(0016148) Torr Samaho (administrator) 2016-11-06 11:31	As unknownna suggested, I restored the option to use ping based unlagged for now. Quote from JKist3 noob question: how do I change between tic based and ping based unlag? The CVAR to toggle this is cl_ping_unlagged. Quote from JKist3 Also just so I completely understand, what exactly does unlaggedactor.wad do/show? For every client, the server spawns debug actors that show where the other players are moved for determining hits of attacks from that client with unlagged. Of course, you are seeing the unlagged information "from the past" with this, but it should give you an idea by how much players are moved for determining hits.

(0016837) StrikerMan780 (reporter) 2017-02-11 19:06 edited on: 2017-02-13 18:01	Yeah, it's definitely best to keep in mind the debug actors are going to lag behind due to latency. Some people back when unlagged was being tested didn't realize this. Seems a lot of bugs were introduced by the tic buffer, because before it became a thing, Gametic unlagged was pretty much perfect. By the way, shouldn't this be on the roadmap?

(0016861) Torr Samaho (administrator) 2017-02-13 18:59	Quote from StrikerMan780 Seems a lot of bugs were introduced by the tic buffer, because before it became a thing, Gametic unlagged was pretty much perfect. The way unlagged is perceived seems to be very different from player to player. Although, the memory of how well / bad unlagged worked in the past compared to now does not seem to be very objective. In any case, the debug builds allow to turn off the tic buffer with "sv_useticbuffer 0". So, for a fair comparison to check whether something is caused by the tic buffer, you should compare the situation with the same 3.0 build using "sv_useticbuffer 0" and "sv_useticbuffer 1". Quote from StrikerMan780 By the way, shouldn't this be on the roadmap? Yes, it should. Since it's not a 3.0 regression, I'll add 3.1 as target.

(0016865) StrikerMan780 (reporter) 2017-02-14 23:29 edited on: 2017-02-14 23:30	I dunno, Ping unlagged pretty often missed direct shots in instagib where Gametic didn't for the majority of players in testing. (Ping also hit when it should have missed, like shots too far ahead of target's direction of movement) I'd say that's fairly objective.

(0018289) JKist3 (reporter) 2017-09-12 11:48 edited on: 2017-09-12 11:49	unknownna: can you test this again on zan 3.0? I just did and was unable to get gametic unlagged to desync at all on a local server.

(0018586) Leonard (developer) 2017-10-22 13:58	I can reproduce this in 3.0. On top of that, switching cl_ping_unlagged on and off clearly shows the desync only happens with gametic unlagged.

(0018597) JKist3 (reporter) 2017-10-22 20:05	How are you testing this Leonard? are you using a local server? I will try to reproduce your steps if you tell me how you are doing it.

(0018627) Leonard (developer) 2017-10-27 21:33 edited on: 2017-11-10 16:31	This was a nightmare to debug. There were actually multiple issues getting in the way of this, both from the ticbuffer and unlagged. After finally fixing everything, I will now create separate tickets and make pull requests for them one by one to keep things simple. I will address this ticket once enough has been fixed and we can clearly see what happens. It's worth noting that while debugging/testing each issue I used this commit which gave me very useful information and also a test wad that goes with it. In the test wad there is a very useful map that allows easy stress-testing of unlagged. This map along with the commit will allow us to gather empirical evidence of what works properly or not. I will refer to this note in the coming tickets whenever it's needed to test the issues. EDIT: Changed the commit's link (it's in a closed branch now so it doesn't show up but you should still be able to pull it directly if you want).

(0019179) Leonard (developer) 2018-04-30 15:54	The relevant issues should be fixed now.

(0019220) StrikerMan780 (reporter) 2018-05-07 20:17	I tried some instagib with a few people, and I didn't get any dropped shots or obvious cases of missing or hitting when I shouldn't have. But, shall I try again when the interpolation is fixed?

(0019224) Leonard (developer) 2018-05-08 09:55	If you wish but keep in mind this will have to be tested on a bigger scale anyways.

(0019397) unknownna (updater) 2018-08-25 15:57 edited on: 2018-09-17 01:22	It seems to have fixed most of the issues, but I noticed that the gametic-unlagged now sometimes take a longer time to readjust itself with ping spikes compared to before, for instance when dropping from 300 ping to 30. It takes around 4-5 seconds before the debug actor "catches up" to the lower ping. Though it seems to be mostly visual, as the unlagged itself seems unaffected, meaning that the target doesn't shift positions during the readjustment. And once it's desynced, it can at worst break it permanently, as the debug actor can then start to flicker behind the player instead of being solid. This only happened once so far for me, and I wasn't able to reproduce it yet. Another thing I noticed is that the debug actors will then still have different distances between the ping- and gametic-unlagged. The debug actor is then further behind again with gametic-unlagged. Actually, this also happens all the time with low latency (13 ping). But, after testing with firing at the target, the ping-unlagged is actually wrong here, as I'm able to fire considerably far ahead of the target in thin air and still hit the player. That's what I found so far. Great work by the way, amazing work! The unlagged definitely feels a lot better and "snappier" after testing with some bots on a server. But the ping-unlagged seems to be broken though at low latencies. Edit: Quote from unknownna * Desyncs after changemap map changes. * Desyncs when you join the game. * Desyncs a little consistently every 24-25 seconds. * Desyncs when respawning. * Desyncs after map resets. I still seem to be getting some form of these issues now for some reason when using gametic-unlagged. The gametic-unlagged breaks for one of the clients permanently and all the debug actors flicker behind every single other player. Will test further. One client or a bot can cause the unlagged to desync or act up for all the other clients. It seemingly goes away when the offending client/bot is removed from the server, and returns when said client/bot returns to the server. * It can either break by chance(?) for a local client and other clients by connecting or adding bots to the server. Removing the offending client/bot temporarily fixes the issue until said client/bot returns to the server. This seems to be a bug and is seemingly permanent. * It can break by connecting to the server through a proxy without any ping emulation aside from the hardcoded one (about 2 ms according to Gamer's Proxy). I saw some very bad desync here where both clients disagreed where they were located on the map with unlagged_debug.wad. But this also happens in 3.0, from a quick test. * The ping unlagged seems to either be broken or non-existent at low latencies sometimes, allowing you to hit up to a half player body distance ahead of the player. This however also happens in 3.0, but I wasn't able to reproduce it consistently yet. * Other players are sometimes moving incredibly jittery between 10 and 17 ping specifically, but after testing it again for a second time trying to reproduce it reliably it works fine. Weird. Managed to get it again and this time the other player was jittery between 0 and 7 ping. So the clients become up to 7-8 ms behind/ahead of the server once it breaks? * When it sometimes breaks, it can delay every command from your client by approx. 800-1000 ms on the server. For instance, you have 160 ping, but your projectiles, puffs, item pickups and player movement to other players appear to take effect only 800-1000 ms later instead of 160. I've observed this particular desync a few times by now. It seems to happen after ping spikes, map changes and map resets in particular. It can correct itself after a few minutes, but not always, and the delay is not necessarily always 800-1000 ms, it can sometimes approx. be 300-600 too. Slightly less, but still higher than your real ping. * The client tends to get stuck in "Connection interrupted!" states after map changes, though it was already reported here: 0003430: [3.1 beta] Connection Interrupted after a few map changes * The unlagged seems to act odd when firing immediately upon passing and bumping into another player close by, allowing you to seemingly hit behind the player in thin air and still hit. But IIRC, it's always been this way since Skulltag. * Other players seem to move less smooth in general if playing with cl_capfps 0 (probably due to the tickrate now being forced to 35hz?). There's something that causes it to break sometimes for sure, but I haven't found out exactly what, but it might have to do something with connecting to the server with very small and random ping fluctuations or something. There's something really weird going on here.

(0021077) StrikerMan780 (reporter) 2019-12-04 06:55	Keep in mind the debug actor isn't the most accurate representation of where unlagged will put someone at a specific given time. It's always going to be behind a bit for the client. It's mostly there to make sure it isn't behaving erratically.

(0024140) unknownna (updater) 2024-11-15 06:45	All the main issues here are fixed after testing the new clock offset fixes and the implementation of the movement buffer. Some issues remain in their own tickets and should be fixed there. It should be safe to remove the ping-based unlagged now if wanted, and from a quick test, it seems to be completely broken with the new behavior, presumably due to the clock offset being shifted. Quote from unknownna When it sometimes breaks, it can delay every command from your client by approx. 800-1000 ms on the server. For instance, you have 160 ping, but your projectiles, puffs, item pickups and player movement to other players appear to take effect only 800-1000 ms later instead of 160. I've observed this particular desync a few times by now. It seems to happen after ping spikes, map changes and map resets in particular. It can correct itself after a few minutes, but not always, and the delay is not necessarily always 800-1000 ms, it can sometimes approx. be 300-600 too. Slightly less, but still higher than your real ping. This sounds like the same issues that StrikerMan780 had reported in'https://zandronum.com/tracker/view.php?id=3314#c19255 [^]' , but it has since been fixed for many years now. And with that, I'm finally marking this as resolved after all these years. Special thanks to Leonard and Kaminsky for their incredible hard work and fixes.

This issue is already marked as resolved. If you feel that is not the case, please reopen it and explain why.
Supporters:	unknownna FascistCat Combinebobnt
Opponents:	No one explicitly opposes this issue yet.

Issue History
Date Modified	Username	Field	Change
2016-10-06 05:39	unknownna	New Issue
2016-10-06 05:39	unknownna	File Added: unlaggedactor.wad
2016-10-06 05:42	unknownna	File Added: GameticVSPing.png
2016-10-06 05:48	unknownna	Steps to Reproduce Updated	View Revisions
2016-10-08 04:31	unknownna	Note Added: 0015790
2016-10-08 04:31	unknownna	Priority	high => urgent
2016-10-08 10:39	unknownna	Note Edited: 0015790	View Revisions
2016-10-11 20:12	unknownna	Note Added: 0015907
2016-10-11 20:17	unknownna	Note Edited: 0015907	View Revisions
2016-10-11 20:26	unknownna	Note Edited: 0015907	View Revisions
2016-10-11 20:53	unknownna	Note Edited: 0015907	View Revisions
2016-10-11 21:02	unknownna	Note Edited: 0015907	View Revisions
2016-10-12 05:39	unknownna	Note Edited: 0015907	View Revisions
2016-10-12 06:17	AlexMax	Note Added: 0015915
2016-10-12 16:27	Ru5tK1ng	Note Added: 0015924
2016-10-14 13:29	unknownna	Relationship added	related to 0002712
2016-10-14 13:47	unknownna	Note Added: 0015988
2016-10-14 13:52	unknownna	Note Edited: 0015988	View Revisions
2016-10-14 13:52	unknownna	Note Edited: 0015988	View Revisions
2016-10-27 22:18	JKist3	Note Added: 0016097
2016-11-06 11:31	Torr Samaho	Note Added: 0016148
2017-02-11 19:06	StrikerMan780	Note Added: 0016837
2017-02-13 18:01	StrikerMan780	Note Edited: 0016837	View Revisions
2017-02-13 18:59	Torr Samaho	Note Added: 0016861
2017-02-13 18:59	Torr Samaho	Target Version	=> 3.1
2017-02-14 23:29	StrikerMan780	Note Added: 0016865
2017-02-14 23:30	StrikerMan780	Note Edited: 0016865	View Revisions
2017-09-12 11:48	JKist3	Note Added: 0018289
2017-09-12 11:49	JKist3	Note Edited: 0018289	View Revisions
2017-10-22 13:58	Leonard	Note Added: 0018586
2017-10-22 13:58	Leonard	Status	new => confirmed
2017-10-22 20:05	JKist3	Note Added: 0018597
2017-10-23 14:50	Leonard	Assigned To	=> Leonard
2017-10-23 14:50	Leonard	Status	confirmed => assigned
2017-10-27 21:33	Leonard	Note Added: 0018627
2017-10-27 21:36	Leonard	File Added: unlagged_debug.wad
2017-10-27 21:39	Leonard	Note Edited: 0018627	View Revisions
2017-11-08 00:59	Leonard	Relationship added	child of 0003334
2017-11-08 00:59	Leonard	Relationship added	child of 0003317
2017-11-08 00:59	Leonard	Relationship added	related to 0003314
2017-11-08 01:00	Leonard	Relationship replaced	child of 0003314
2017-11-08 01:01	Leonard	Relationship added	child of 0003316
2017-11-10 16:29	Leonard	Note Edited: 0018627	View Revisions
2017-11-10 16:31	Leonard	Note Edited: 0018627	View Revisions
2018-04-30 15:54	Leonard	Note Added: 0019179
2018-04-30 15:54	Leonard	Status	assigned => needs testing
2018-05-07 20:17	StrikerMan780	Note Added: 0019220
2018-05-08 09:55	Leonard	Note Added: 0019224
2018-08-25 15:57	unknownna	Note Added: 0019397
2018-08-25 16:16	unknownna	Note Edited: 0019397	View Revisions
2018-08-25 16:23	unknownna	Note Edited: 0019397	View Revisions
2018-08-25 16:24	unknownna	Note Edited: 0019397	View Revisions
2018-08-26 16:48	unknownna	Note Edited: 0019397	View Revisions
2018-08-26 17:00	unknownna	Note Edited: 0019397	View Revisions
2018-08-26 17:10	unknownna	Note Edited: 0019397	View Revisions
2018-08-26 17:25	unknownna	Note Edited: 0019397	View Revisions
2018-08-28 05:14	unknownna	Note Edited: 0019397	View Revisions
2018-08-28 05:50	unknownna	Note Edited: 0019397	View Revisions
2018-08-28 05:50	unknownna	Note Edited: 0019397	View Revisions
2018-08-28 06:16	unknownna	Note Edited: 0019397	View Revisions
2018-09-09 07:23	unknownna	Note Edited: 0019397	View Revisions
2018-09-17 01:22	unknownna	Note Edited: 0019397	View Revisions
2019-12-04 06:55	StrikerMan780	Note Added: 0021077
2024-03-07 09:24	unknownna	Relationship added	related to 0004202
2024-11-15 06:45	unknownna	Note Added: 0024140
2024-11-15 06:45	unknownna	Status	needs testing => resolved
2024-11-15 06:45	unknownna	Fixed in Version	=> 3.2
2024-11-15 06:45	unknownna	Resolution	open => fixed