MantisBT - Zandronum
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0001542Zandronum[All Projects] Suggestionpublic2013-10-15 02:282014-06-15 14:34
ReporterAlexMax 
Assigned To 
PrioritynormalSeverityminorReproducibilityN/A
StatusacknowledgedResolutionopen 
PlatformOSOS Version
Product Version1.2 
Target VersionFixed in Version 
Summary0001542: Master/Server Query Protocol improvements.
DescriptionThe master server protocol seems to make a pretty fundamental assumption about the ordering of packets that I believe does not exactly hold. When getting the initial list of servers, it appears that you get a packet number with each piece of the master list, along with a signal of if this is the 'last' packet or if there are more servers to deliver.

This setup makes it more difficult than it needs to be to ensure that you have all the necessary packets for a complete view of servers. UDP has neither guaranteed delivery or guaranteed sequence, and with the current setup you have to do a little bit of inferring to determine if you have a complete view of the server list, which becomes much more difficult if the last master response packet gets dropped (because then you can't know how many pieces you're missing).

Thus, I would like to suggest an alternative. I think that it would be a wise idea for each one of the master response packets to have three items in the header:

- Truncated timestamp that the packet assembly _started_ at.
- Current packet
- Total number of packets.

The truncated timestamp would behave as a unique identifier for a group of responses, and being told how many packets to expect ahead of time makes it trivial to figure out how much of the server list you actually got and how many packets got through or were lost.

You could potentially also extend this system to server query response packets, since the current system seems to rely on you only requesting specific subsets of data.

There are two other random changes that I think might be worthwhile.

First is that I believe that we can save a few bytes by changing the master response 'packet type flag' to a byte instead of a long. There aren't enough possible responses to warrant it being a long.

Second is to abandon huffman coding and go with a fast compression scheme such as lz4. Initial tests of mine reveal it cutting bandwidth usage in half, though I need to run some real tests.
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0000892new  Master server user agent 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2013-10-15 02:28AlexMaxNew Issue
2013-10-15 12:59Blzut3Note Added: 0007415
2013-10-15 16:27AlexMaxNote Added: 0007416
2013-10-15 16:30Blzut3Relationship addedrelated to 0000892
2014-06-15 14:34WatermelonStatusnew => acknowledged

Notes
(0007415)
Blzut3   
2013-10-15 12:59   
Although I can see how one might get a minor improvement out of knowing the number of packets before hand. The method to determine if a transmission is complete isn't all that complicated. Maintain a list of what packets you received and then note the packet number of the terminator when you get it. Until then you know you need at least n+1 where n is the highest numbered packet.

Is there a particular reason a timestamp would make things easier. I can't really think of a reason one would need to differentiate between multiple simultaneous transmissions from the master unless your program is malformed.
(0007416)
AlexMax   
2013-10-15 16:27   
I did not say it was hard, merely that it was more difficult than it needed to be and also not stateless. Currently, there are three possible outcomes from the current master query protocol:

1. You get all packets. Life is good.
2. You are missing a packet in the middle. You know that the server list is truncated, but can't do anything about it except wait for the timeout to pass and request the entire server list again.
3. You are missing the terminator packet. You know that the server list is truncated, but can't do anything about it and also have no idea how many servers you are missing. You must wait for the timeout to pass and request the entire server list again.

When you have additional headers and a unique identifier, suddenly this opens up possibilities. The server browser can know for certain which packets it has and which it doesn't. It can then - instead of requesting the entire server list again - request JUST the data it was missing by sending it the identifier and the list of missing packets (or maybe even server ids). You could have these retransmit requests bypass the 3 second timeout and be throttled in their own way. Perhaps you could be even smarter about the whole thing and split the retransmitted packet into two pieces under the assumption that perhaps a router between the master and the client dropped the packet because it was too big (it seems like there is a 1024 byte limit right now, which can be too generous in some cases).