Zandronum Chat on our Discord Server Get the latest version: 3.1
Source Code

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0001954Zandronum[All Projects] Suggestionpublic2014-10-10 15:282016-04-10 19:27
ReporterWatermelon 
Assigned To 
PrioritynormalSeveritytweakReproducibilityN/A
StatusnewResolutionsuspended 
PlatformOSOS Version
Product Version1.3 
Target VersionFixed in Version 
Summary0001954: Changing sound sending from String to logical index to save bandwidth
DescriptionRight now all the sound functions communicate via String. Therefore (excluding compression) we'd be sending stuff similar to 'myactor/attacksound123' over the wire rather than it's index (like 155).

I noticed while playing ZDoomWars, when I edited out the sounds, the bandwidth had a significant decrease if they were played a lot. After all, 23 bytes vs 2 is a big chunk when sent a lot.

I figure sending the logical index as a unsigned short and letting the client look it up should be good enough. Another option for unknown sounds could be to send 65535 and then followed by the custom string if it's unknown, or to not send the sound if it's not indexed as a logical sound.
Attached Files

- Relationships
related to 0001473closedDusk "Optional Wads" for servers 
related to 0000757new Monster Bandwidth Mitigation 

-  Notes
User avatar (0010456)
Watermelon (developer)
2014-10-10 17:13

'https://bitbucket.org/ChrisKOmg/zandronum-stable/commits/f0303bc828a6f067b7047ed2186c6da0be597dff [^]'


I tested this and the client functions just as normal, but with much less bandwidth when there's tons of weapons being fired.

I could extend it to others, but these were the main culprits of bandwidth hogging.

Can someone code review this to make sure I'm not missing anything?
User avatar (0010461)
Blzut3 (administrator)
2014-10-11 01:19

I don't believe we make any guarantee that the sound indexes are the same (would require SNDINFO to be protected, which in turn would make skins problematic). That said, I think that only raises a concern if anything extra is loaded before the mods loaded by the server. Or if for some reason someone modified their zandronum.pk3.

Could have some implications for 0001473 as optional wads would need to be loaded by the server following those required.

Requiring proper load order could be a good thing as it could potentially also imply that the FName table is safe as well.
User avatar (0010462)
Watermelon (developer)
2014-10-11 03:15

One solution to this if it fails I'm guessing would be to make a custom array for required sounds and send from that.

I'm going to investigate how skins handle sound overriding.
User avatar (0010467)
Torr Samaho (administrator)
2014-10-11 07:58

Yeah, I'm rather sure that with the current possibilities of the skins directory you cannot assume that the sound indices are equal on the server and the clients. The files from the skins dir are loaded before the files specified with "-file".
User avatar (0010468)
Edward-san (developer)
2014-10-11 08:47

I wonder if you can do a reverse indexing: last has index 1, first without skins dir has last index.
User avatar (0010470)
Torr Samaho (administrator)
2014-10-11 08:51

This wouldn't help: While skins are loaded before "-file" stuff, they are loaded after zandronum.pk3 and the IWAD.
User avatar (0010478)
Watermelon (developer)
2014-10-11 12:45
edited on: 2014-10-11 12:45

I'm going to have to create an indexed list that doesn't take into account the skins folder.

My plan is to not write any SNDINFO definitions from files that aren't required (this way it can be expanded to ignore optional wads too when added).

User avatar (0010743)
Watermelon (developer)
2014-10-30 22:58
edited on: 2014-10-31 03:21

My idea right now is as follows (using Dusk's idea):

1) Index all the sounds in a TArray based on their index
2) Send the index and sound over to the client
3) Mark in a boolean list which have been sent (or some kind of hashmap)
4) Only send the sound ID when it has been sent once
5) Clear on client disconnecting

This way we get around the optional wad and sound order problem.

User avatar (0010753)
Watermelon (developer)
2014-10-31 23:36

Requesting review:

'https://bitbucket.org/ChrisKOmg/zandronum_stable/commits/6b8e9408c28d8dc012620e8895743e5728c39161 [^]'

'https://bitbucket.org/ChrisKOmg/zandronum_stable/commits/49a37ba6efe65f4b6a0b9263fdca8decc0a37ce7?at=sound_bandwidth_compression [^]'
User avatar (0010756)
Blzut3 (administrator)
2014-11-01 05:03

Wouldn't it be possible to do the sound name lookup before storing into g_sndinfoLookupTable? That way it just translates a server index to a client index.
User avatar (0010759)
Torr Samaho (administrator)
2014-11-01 11:29

First, there is lots of copy and paste in here, which is always bad :P. Second, trying to manually maintain a list of used sounds for each client is quite cumbersome. Can't you just send the full table when a client connects? Or is the table too big for this?
User avatar (0010762)
Dusk (developer)
2014-11-01 11:59

Quote
Can't you just send the full table when a client connects?

I'm a bit worried this might be a rather large bandwidth sink.
User avatar (0010763)
Torr Samaho (administrator)
2014-11-01 12:17

Did you check how big the table is? You may be overestimating its size.
User avatar (0010765)
Watermelon (developer)
2014-11-01 14:32

Quote
First, there is lots of copy and paste in here, which is always bad :P.


Are we okay with macros? That'd cut down the annoying code duplication.
User avatar (0010766)
Torr Samaho (administrator)
2014-11-01 15:53

I'd say macros are a last ditch effort to reduce code duplications, where functions and classes simply can't do it. But I don't see why you can't put

    // [CK] Now read the short and bitmask.
    sSoundID = NETWORK_ReadShort( pByteStream );
    
    // [CK] If the server is sending us a new sound, read the string, otherwise
    // we will get it from our indexed sound map.
    if ( sSoundID & SOUNDID_CONTAINS_STRING_MASK )
    {
        pszSound = NETWORK_ReadString( pByteStream );
        CLIENT_AddToSndinfoTable( sSoundID & SOUNDID_NETWORK_ID_MASK , pszSound );
    }
    else
    {
        pszSound = CLIENT_GetSoundStringFromSndinfoTable( sSoundID );
    }

into a function that gets pByteStream and returns pszSound.

But the code fragment above is not necessary anyway, if the whole table is transferred on connect. So, before just reducing the code duplication, how big is the table?
User avatar (0010768)
Watermelon (developer)
2014-11-01 17:20

So far on IRC we estimated it to be around 20 kb at worst, on average most mods are 10-12 kb.
User avatar (0010769)
Watermelon (developer)
2014-11-01 17:34
edited on: 2014-11-01 17:34

On further thought, I'm going to try out the "send the full table" idea since in addition to what you said, it will integrate *way* easier with Zan 2.0's netcommandification (and then I don't have to do these annoying copy paste things)

User avatar (0010771)
Torr Samaho (administrator)
2014-11-01 17:37

Quote from Watermelon

So far on IRC we estimated it to be around 20 kb at worst, on average most mods are 10-12 kb.

That amount shouldn't be any problem if it's only sent when initially connecting to the server.
User avatar (0010780)
Edward-san (developer)
2014-11-01 19:25
edited on: 2014-11-01 19:25

Is is there a hardcoded limit for the number of sounds per zdoom game? It's in order to get the number of bytes to assign to the index.

User avatar (0010781)
Watermelon (developer)
2014-11-01 19:49

So far the idea is to allow 32767 sounds, which no wad should exceed hopefully ever.


If they ever did, then we can always tweak it to make 0x7FFF represent a raw string.
User avatar (0010798)
Watermelon (developer)
2014-11-02 20:32

Pull request with a table sent on connection done.
User avatar (0010818)
Torr Samaho (administrator)
2014-11-03 19:52

I left some comments in the pull request.
User avatar (0010850)
Watermelon (developer)
2014-11-10 03:24

Submitted updated pull request.
User avatar (0010936)
Torr Samaho (administrator)
2014-11-23 11:07
edited on: 2014-11-23 11:10

You may have overlooked my question in the pull request:
Quote from Torr Samaho

You didn't post the pull request for the NetCommand rewrite of the SERVERCOMMANDS_* functions in question, did you?


User avatar (0010949)
Watermelon (developer)
2014-11-24 12:35

The commit is in the zandronum branch here:
'https://bitbucket.org/Torr_Samaho/zandronum/commits/cd2f520a94dac386ca7952fa6243b57a64cd095a [^]'

Am I supposed to pull this into the sandbox, and then get you to pull this? Wouldn't that create a possible merge headache for you when merging into zandronum branch?
User avatar (0010961)
Torr Samaho (administrator)
2014-11-24 20:51

No, you misunderstood me. I'm not talking about upgrading all SERVERCOMMANDS_* functions in 1.4. Let me dig out my old comment from the pull request that is hidden by now:
Quote from Torr Samaho

The easiest thing would be if you first send a pull request for the stable repository that just upgrades the SERVERCOMMANDS_* functions necessary for the sound table changes to their 2.0 versions. Then I can pull the changes and merge them into the main repository. Finally, you can send a new pull request with the actual sound info changes.
User avatar (0010963)
Watermelon (developer)
2014-11-24 21:40
edited on: 2014-11-24 21:42

I'm sorry if this isn't making sense. Can I get some further clarification? Based on what I read, if I interpreted this right:



Quote
The easiest thing would be if you first send a pull request for the stable repository that just upgrades the SERVERCOMMANDS_* functions necessary for the sound table changes to their 2.0 versions


If I read this right, send a pull request from the sandbox-zandronum-stable repository to the 2.0 repository that has the new SERVERCOMMANDS update. Then somehow select from that pull only the lines that we want that will be applied.

Then you pull the changes into zandronum-stable from the sandbox repo from a pull request I make based off of this. After that I then send just the sound info pull request based on the SERVERCOMMANDS update.



I think I'm getting confused by your first line.
If I get this wrong, could you restate it (maybe something like: sandbox pulls from 2.0 branch <something>, then you do <something>, then sandbox makes a pull request for stable.

User avatar (0010995)
Torr Samaho (administrator)
2014-11-29 07:40

Sorry, for the unclear wording. Let me elaborate. First you need to manually copy the NetCommand based implementation of the SERVERCOMMANDS_* functions necessary for the sound table changes from 2.0 to your stable head (replacing their old versions in the stable head) and commit this. Then send a pull request for this changeset only to the stable repo. I'll then pull this changeset to stable and merge it with 2.0 (which should actually mean no changes and no conflicts during the merge, because the changes originate from 2.0). Then you can make a pull request for the traffic optimization based on the stable version that already has the necessary upgraded SERVERCOMMANDS_* functions.
User avatar (0011046)
Watermelon (developer)
2014-12-08 15:22

Updates done.
User avatar (0011091)
cobalt (updater)
2014-12-26 09:56

Issue addressed by commit a15cb89a8f98: Changed sound transmission from a string to a lookup table short, thus saving a Significant amount of bandwidth (fixes 1954).
Committed by ChrisKOmg [Watermelon] on Monday 08 December 2014 23:59:21

Changes in files:
 docs/zandronum-history.txt | 1 +
 src/cl_main.cpp | 61 ++++++++++++++++++++++++++++++++++++++++++++++++-------------
 src/network_enums.h | 1 +
 src/sv_commands.cpp | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------
 src/sv_commands.h | 1 +
 src/sv_main.cpp | 4 ++++
 6 files changed, 122 insertions(+), 19 deletions(-)
User avatar (0011246)
Torr Samaho (administrator)
2015-01-03 19:16

Apparently, the current approach of sending the table is infeasible, see 0002045. Either we need to fix this or completely remove it.
User avatar (0011271)
Torr Samaho (administrator)
2015-01-04 12:47

To not stall the release of 2.0, we decided to remove this for the time being. A proper way to fix this, is to introduce output packet buffers that allow us to control how many packets per time are send to an individual client. This will also solve other problems. Once we have those, we can re-introduce the sound table optimization.

Issue Community Support
Only registered users can voice their support. Click here to register, or here to log in.
Supporters: Konar6 Ivan Combinebobnt Argentum EnsaladaDeTomate
Opponents: No one explicitly opposes this issue yet.

- Issue History
Date Modified Username Field Change
2014-10-10 15:28 Watermelon New Issue
2014-10-10 15:28 Watermelon Status new => assigned
2014-10-10 15:28 Watermelon Assigned To => Watermelon
2014-10-10 16:01 Watermelon Relationship added related to 0000757
2014-10-10 17:13 Watermelon Note Added: 0010456
2014-10-10 17:13 Watermelon Status assigned => needs review
2014-10-11 01:19 Blzut3 Note Added: 0010461
2014-10-11 03:15 Watermelon Note Added: 0010462
2014-10-11 03:21 Watermelon Relationship added related to 0001473
2014-10-11 07:58 Torr Samaho Note Added: 0010467
2014-10-11 07:58 Torr Samaho Status needs review => new
2014-10-11 07:58 Torr Samaho Status new => feedback
2014-10-11 08:47 Edward-san Note Added: 0010468
2014-10-11 08:51 Torr Samaho Note Added: 0010470
2014-10-11 12:44 Watermelon Status feedback => assigned
2014-10-11 12:45 Watermelon Note Added: 0010478
2014-10-11 12:45 Watermelon Note Edited: 0010478 View Revisions
2014-10-30 22:58 Watermelon Note Added: 0010743
2014-10-31 03:21 Watermelon Note Edited: 0010743 View Revisions
2014-10-31 23:36 Watermelon Note Added: 0010753
2014-10-31 23:36 Watermelon Status assigned => needs review
2014-11-01 05:03 Blzut3 Note Added: 0010756
2014-11-01 11:29 Torr Samaho Note Added: 0010759
2014-11-01 11:29 Torr Samaho Status needs review => feedback
2014-11-01 11:59 Dusk Note Added: 0010762
2014-11-01 12:17 Torr Samaho Note Added: 0010763
2014-11-01 14:32 Watermelon Note Added: 0010765
2014-11-01 14:32 Watermelon Status feedback => assigned
2014-11-01 15:53 Torr Samaho Note Added: 0010766
2014-11-01 17:20 Watermelon Note Added: 0010768
2014-11-01 17:34 Watermelon Note Added: 0010769
2014-11-01 17:34 Watermelon Note Edited: 0010769 View Revisions
2014-11-01 17:37 Torr Samaho Note Added: 0010771
2014-11-01 19:25 Edward-san Note Added: 0010780
2014-11-01 19:25 Edward-san Note Edited: 0010780 View Revisions
2014-11-01 19:49 Watermelon Note Added: 0010781
2014-11-02 20:32 Watermelon Note Added: 0010798
2014-11-02 20:32 Watermelon Status assigned => needs review
2014-11-03 19:52 Torr Samaho Note Added: 0010818
2014-11-03 19:53 Torr Samaho Status needs review => feedback
2014-11-10 03:24 Watermelon Note Added: 0010850
2014-11-10 03:24 Watermelon Status feedback => assigned
2014-11-10 03:24 Watermelon Status assigned => needs review
2014-11-23 11:07 Torr Samaho Note Added: 0010936
2014-11-23 11:07 Torr Samaho Status needs review => feedback
2014-11-23 11:10 Torr Samaho Note Edited: 0010936 View Revisions
2014-11-24 12:35 Watermelon Note Added: 0010949
2014-11-24 12:35 Watermelon Status feedback => assigned
2014-11-24 12:35 Watermelon Status assigned => needs review
2014-11-24 20:51 Torr Samaho Note Added: 0010961
2014-11-24 20:53 Torr Samaho Status needs review => feedback
2014-11-24 21:40 Watermelon Note Added: 0010963
2014-11-24 21:40 Watermelon Status feedback => assigned
2014-11-24 21:40 Watermelon Status assigned => needs review
2014-11-24 21:41 Watermelon Note Edited: 0010963 View Revisions
2014-11-24 21:42 Watermelon Note Edited: 0010963 View Revisions
2014-11-29 07:40 Torr Samaho Note Added: 0010995
2014-11-29 07:41 Torr Samaho Status needs review => feedback
2014-12-01 12:59 Watermelon Status feedback => needs review
2014-12-08 15:22 Watermelon Note Added: 0011046
2014-12-26 09:56 cobalt Status needs review => needs testing
2014-12-26 09:56 cobalt Description Updated View Revisions
2014-12-26 09:56 cobalt Note Added: 0011091
2015-01-03 19:14 Torr Samaho Product Version 1.4 => 1.3
2015-01-03 19:14 Torr Samaho Description Updated View Revisions
2015-01-03 19:16 Torr Samaho Note Added: 0011246
2015-01-03 19:16 Torr Samaho Status needs testing => feedback
2015-01-04 12:47 Torr Samaho Note Added: 0011271
2015-01-04 12:48 Torr Samaho Target Version 1.4 =>
2015-01-05 20:09 Dusk Status feedback => acknowledged
2015-01-05 20:09 Dusk Resolution open => suspended
2015-02-22 18:13 Watermelon Status acknowledged => assigned
2016-04-10 19:25 Dusk Status assigned => new
2016-04-10 19:27 Dusk Assigned To Watermelon =>






Questions or other issues? Contact Us.

Links


Copyright © 2000 - 2024 MantisBT Team
Powered by Mantis Bugtracker