MantisBT - Zandronum
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0001473Zandronum[All Projects] Suggestionpublic2013-08-21 23:282018-09-30 22:57
ReporterToxicity 
Assigned ToDusk 
PrioritynormalSeverityfeatureReproducibilityN/A
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target Version2.0Fixed in Version2.0 
Summary0001473: "Optional Wads" for servers
DescriptionOptional wads, such like ZDaemon has.

When setting up the server, there can be an extra box for adding "optional" wads, such as skins, announcers, music packs, and HUDs. Trying to add an optional wad with protected lumps will result in the wad being rejected upon startup. (An error like "OPTIONAL WAD ______ COULD NOT BE ADDED - Only non-protected lumps are allowed" would be appropriate.)

Clients viewing the server from Doomseeker, in addition to seeing what mandatory wads a server uses (like they currently do) will also see any optional wads listed right under that. When GetWad opens to download the mandatory wads, it will prompt the user "file _____ is optional, download it?".

And then the client connects. The server runs ALL the wads (as if they were all added normally), but the clients are not required to. Pretty straightforward.
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
parent of 0002182closed Dusk -optfile thinks .cfg files are optional wads 
related to 0001954new  Changing sound sending from String to logical index to save bandwidth 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2013-08-21 23:28ToxicityNew Issue
2013-08-21 23:32ToxicityNote Added: 0007055
2013-08-22 04:22EmpyreNote Added: 0007057
2013-08-22 05:47Blzut3ProjectDoomseeker => Zandronum
2013-08-22 05:49Blzut3Note Added: 0007058
2013-08-22 08:55DuskDescription Updatedbug_revision_view_page.php?rev_id=3962#r3962
2013-08-22 20:00ToxicityNote Added: 0007059
2013-08-27 20:32BalrogNote Added: 0007074
2013-08-28 14:53ToxicityNote Added: 0007076
2013-08-31 00:06BalrogNote Added: 0007087
2013-08-31 00:07BalrogNote Edited: 0007087bug_revision_view_page.php?bugnote_id=7087#r3973
2013-09-05 10:44ToxicityNote Added: 0007109
2013-09-05 10:46ToxicityNote Edited: 0007109bug_revision_view_page.php?bugnote_id=7109#r3992
2013-09-05 13:49someoneelseNote Added: 0007111
2013-09-05 15:10DuskNote Added: 0007115
2013-09-05 15:11DuskNote Edited: 0007115bug_revision_view_page.php?bugnote_id=7115#r3996
2013-09-05 15:12DuskAssigned To => Dusk
2013-09-05 15:12DuskStatusnew => assigned
2013-09-05 20:01EmpyreNote Added: 0007117
2013-09-05 20:24DuskNote Added: 0007118
2013-09-06 19:36ToxicityNote Added: 0007124
2013-09-06 20:34DuskNote Deleted: 0007124
2013-09-27 10:39DuskStatusassigned => confirmed
2013-09-27 10:39DuskStatusconfirmed => new
2013-09-27 23:50ToxicityNote Added: 0007291
2013-09-28 08:16DuskNote Added: 0007292
2013-09-28 08:16DuskStatusnew => acknowledged
2013-09-28 08:16DuskResolutionopen => suspended
2013-09-28 23:24ToxicityNote Added: 0007293
2014-10-11 03:20WatermelonNote Added: 0010463
2014-10-11 03:20WatermelonNote Edited: 0010463bug_revision_view_page.php?bugnote_id=10463#r5665
2014-10-11 03:20WatermelonNote Edited: 0010463bug_revision_view_page.php?bugnote_id=10463#r5666
2014-10-11 03:21WatermelonRelationship addedrelated to 0001954
2014-10-11 11:46DuskNote Added: 0010476
2014-10-11 11:46DuskResolutionsuspended => open
2014-10-11 11:46DuskTarget Version => 2.0
2014-10-11 12:57WatermelonNote Added: 0010480
2014-10-25 12:42DuskNote Added: 0010683
2014-10-25 12:42DuskStatusacknowledged => assigned
2014-10-25 19:19DuskNote Added: 0010688
2014-10-25 19:19DuskStatusassigned => needs review
2014-10-25 23:45DuskNote Edited: 0010688bug_revision_view_page.php?bugnote_id=10688#r5866
2014-10-27 15:05DuskNote Added: 0010705
2014-11-02 07:26Torr SamahoNote Added: 0010789
2014-11-02 07:26Torr SamahoStatusneeds review => feedback
2014-11-02 15:59DuskNote Added: 0010793
2014-11-02 16:00DuskNote Edited: 0010793bug_revision_view_page.php?bugnote_id=10793#r5920
2014-11-02 16:26Torr SamahoNote Added: 0010794
2014-11-02 19:51DuskNote Added: 0010797
2014-11-02 19:51DuskStatusfeedback => assigned
2014-11-04 01:35DuskNote Added: 0010822
2014-11-04 01:35DuskStatusassigned => needs review
2014-11-04 01:37DuskNote Edited: 0010822bug_revision_view_page.php?bugnote_id=10822#r5935
2014-11-04 20:00Torr SamahoNote Added: 0010829
2015-01-18 08:42cobaltStatusneeds review => needs testing
2015-01-18 08:42cobaltDescription Updatedbug_revision_view_page.php?rev_id=6433#r6433
2015-01-18 08:42cobaltNote Added: 0011425
2015-03-29 20:20DuskStatusneeds testing => resolved
2015-03-29 20:20DuskFixed in Version => 2.0
2015-03-29 20:20DuskResolutionopen => fixed
2015-04-16 12:18DuskRelationship addedparent of 0002182
2018-09-30 22:57Blzut3Statusresolved => closed

Notes
(0007055)
Toxicity   
2013-08-21 23:32   
Oops. When I said "When WadHost opens to download the mandatory wads", I meant "When GetWad opens to download the mandatory wads". :S
(0007057)
Empyre   
2013-08-22 04:22   
I think you can edit your own suggestion.
If this is implemented, both Doomseeker, which uses Wadseeker, and IDE, which uses GetWad would need to be updated.
I support this because it would add functionality that the skins folder can't do, or can do but very awkwardly.
(0007058)
Blzut3   
2013-08-22 05:49   
Doomseeker already supports optional wads when joining. It once had ZDaemon support. :P
(0007059)
Toxicity   
2013-08-22 20:00   
Sooooo ... does that mean it can easily be re-written to work with Zandronum.exe?
(0007074)
Balrog   
2013-08-27 20:32   
You'd need the launcher protocol changed to have an isOptional field for the PWAD list.
(0007076)
Toxicity   
2013-08-28 14:53   
And would that be easy to do?
(0007087)
Balrog   
2013-08-31 00:06   
(edited on: 2013-08-31 00:07)
If you're Torr, yes. If you're anyone else, it's impossible because you'd be breaking compatibility with every other fork of Zandronum.

Looking on the Wiki it wouldn't be too hard to tack a long or string (depends on whether it's considered acceptable to limit the number of possible PWADs to 32 by making it a long at 1bit/wad) onto SQF_PWADS to mark optional wads.
(0007109)
Toxicity   
2013-09-05 10:44   
(edited on: 2013-09-05 10:46)
Well has anyone EVER hosted 32 PWADs at once? I don't think anyone would complain or even notice if you set the PWAD limit to 32 for this purpose.

EDIT:
...Unless /Skins folder wads count towards that limit. Then that might be a problem for some people.
(0007111)
someoneelse   
2013-09-05 13:49   
It hopefully would stop forcing people to download 200MB of music... Sounds awesome.
(0007115)
Dusk   
2013-09-05 15:10   
(edited on: 2013-09-05 15:11)
After some discussion about this in #grandvoid I got a few things to point here:

- Ultimately, it's just a launcher hint, since as far as the actual connection goes you can leave these wads out anyway.
- If the optional wads are loaded last, we don't need a bitfield but just a byte indicating how many of the last wads loaded are optional.

Interface-wise it could go like..
zandronum -host -file awesome_mappack.pk3 -optfile awesome_mappack_music.pk3

-> IWAD: doom2.wad
-> PWADs: awesome_mappack.pk3, [awesome_mappack_music.pk3]

Obviously there needs to be an error/warning if one tries to -optfile a wad that contains protected lumps. I think I could look into this...
(0007117)
Empyre   
2013-09-05 20:01   
I have loaded 33 wads on my server before. Deathwatch and his friends were making a coop mod with lots of player classes, and he decided not to make players download a huge pk3 every time he made a change, so he released smaller incremental pk3 files. It wound up with 30 pk3 files in all. I loaded those plus another for custom monsters, and a wad for my maps, and another for additional maps, for a total of 33 pwads.
How about a boolean for each wad, stored in a byte? It would be less efficient, but unlimited.
(0007118)
Dusk   
2013-09-05 20:24   
Well with the scheme I'm putting together, the only limit is that optional wads are always loaded last and that there's a limit of 255 optional wads (since that's how much a byte can hold). I don't think anybody will need more than that. :P
(0007291)
Toxicity   
2013-09-27 23:50   
I noticed the status changed back to "New". Is there any updates on this? I don't mean to sound impatient, I'm just curious as to what's going on on the development side.
(0007292)
Dusk   
2013-09-28 08:16   
Basically the ZDoom backports are largely conflicting the changes I need to do to make this happen, so I'm holding this off until the backporting is complete.
(0007293)
Toxicity   
2013-09-28 23:24   
Ahh, okay. Thank you for the clarification.
(0010463)
Watermelon   
2014-10-11 03:20   
Important:
Are wads allowed to be loaded anywhere, or will they be strictly at the end? I need to know this for 0001954
(0010476)
Dusk   
2014-10-11 11:46   
Oh yeah this ticket also exists. Now that the WAD issue is out of the way this could be done for 2.0

Water: I don't know how do you go about loading WADs at the end of the loading process. Loading kind of needs WADs to actually load anything, right?
(0010480)
Watermelon   
2014-10-11 12:57   
I found that it doesn't matter if I load it at the end, I have to make a special index list for 0001954 regardless, the only thing this ticket might need to do is mark pre-processing that it's an optional wad somewhere, and after it's done processing turn that mark off so the SNDINFO parser knows to not put any sounds in a list to be sent from server->client.


When the time comes I should be able to add in the code to your patch after you submit it.
(0010683)
Dusk   
2014-10-25 12:42   
Working on this now. I managed to detect when a wad contains protected lumps or maps. Perhaps the server could report all wads that don't contain protected lumps or maps as optional? Or do we need an '-optfile' for this?
(0010688)
Dusk   
2014-10-25 19:19   
(edited on: 2014-10-25 23:45)
Here comes the initial version: f4b0af7e

It's using automatic detection for optionality. Needs a substantial amount of testing to ensure it doesn't flag things incorrectly..

edit: more commits are in
(0010705)
Dusk   
2014-10-27 15:05   
As per discussion, added -optfile and the cvar preferoptionalwads (which causes everything that can be optional to be hinted optional): 0a60c53
(0010789)
Torr Samaho   
2014-11-02 07:26   
Some things I noticed so far:
- How do you ensure that the client knows the order in which the server loaded the wads? For instance, the server loads the wads A, B, C (in that order) and B is automatically detected to be optional. The server then will tell the launcher wads A and C are mandatory, B is optional, but the client has no idea that the order is supposed to be A, B, C.
- I think the way to detect embedded wads could be better separated from ZDoom code. Instead of keeping track of Parent and TopLevelParent, you could just check if the name contains a ':' (like network_InitPWADList is doing). In this case, you know the wad is embedded and just need to identify the containing wad, which is before the colon.
(0010793)
Dusk   
2014-11-02 15:59   
(edited on: 2014-11-02 16:00)
Quote

 - How do you ensure that the client knows the order in which the server loaded the wads? For instance, the server loads the wads A, B, C (in that order) and B is automatically detected to be optional. The server then will tell the launcher wads A and C are mandatory, B is optional, but the client has no idea that the order is supposed to be A, B, C.

The optional wad indices reported by SQF_OPTIONAL_WADS are in the order as reported by SQF_PWADS.

Quote

 - I think the way to detect embedded wads could be better separated from ZDoom code. Instead of keeping track of Parent and TopLevelParent, you could just check if the name contains a ':' (like network_InitPWADList is doing). In this case, you know the wad is embedded and just need to identify the containing wad, which is before the colon.

I found this a rather gross hack and figured to have a clean way of detecting this. Isn't this form of wad name supposed to be used for display purposes? Or is it guaranteed that wad name cannot ever contain a ':'? :-/
(0010794)
Torr Samaho   
2014-11-02 16:26   
Quote from Dusk
The optional wad indices reported by SQF_OPTIONAL_WADS are in the order as reported by SQF_PWADS.

Ah, I see. So the launcher needs to create the -file parameters from this information. Is the compatible with Doomseeker's existing optional wad support?
Quote from Dusk
I found this a rather gross hack and figured to have a clean way of detecting this. Isn't this form of wad name supposed to be used for display purposes? Or is it guaranteed that wad name cannot ever contain a ':'? :-/

I agree that your implementation is nicer, but unfortunately we have to keep ZDoom compatibility in mind. It was a huge pain to port our "loaded automatically" stuff through ZDoom's resource management rewrite. The more custom additions we have in this kind of classes, the more painful it gets. Also, I'm pretty sure that it's impossible under Windows to have ':' in a file name. So yes, I'd say it's safe to assume that a wad name cannot ever contain a ':'.
(0010797)
Dusk   
2014-11-02 19:51   
Fair enough. I'll get to amending this then.
(0010822)
Dusk   
2014-11-04 01:35   
(edited on: 2014-11-04 01:37)
Quote
- I think the way to detect embedded wads could be better separated from ZDoom code. Instead of keeping track of Parent and TopLevelParent, you could just check if the name contains a ':' (like network_InitPWADList is doing). In this case, you know the wad is embedded and just need to identify the containing wad, which is before the colon.

This is now done. It seems to work fine with zpack.pk3.

Quote
So the launcher needs to create the -file parameters from this information. Is the compatible with Doomseeker's existing optional wad support?

To be honest, I have no idea. I don't know how Doomseeker's existing optional wad support works at all, I've never played ZDaemon outside of a few private games. I'm implementing a launcher hint here and figured this would be the most network-efficient way to do it. It's up to Doomseeker to deal with the hint.
(0010829)
Torr Samaho   
2014-11-04 20:00   
Can you get in contact with Blzut3 and discuss how to integrate this with Doomseeker? Without launcher support, we can't do any real testing.
(0011425)
cobalt   
2015-01-18 08:42   
Issue addressed by commit 1a225046b487: - the server can now advertise certain WADs as optional so long as clients are able to join the game without needing said WADs (addresses 1473)
Committed by Teemu Piippo [Dusk] on Saturday 17 January 2015 19:17:54

Changes in files:
 docs/zandronum-history.txt | 1 +
 src/browser.cpp | 9 ++++++++
 src/cl_demo.cpp | 9 +++----
 src/cl_main.cpp | 9 +++++--
 src/d_main.cpp | 8 +++++-
 src/network.cpp | 68 ++++++++++++++++++++++++++++++++++++++++++++-------------------
 src/network.h | 10 ++++++++-
 src/resourcefiles/resourcefile.cpp | 1 +
 src/resourcefiles/resourcefile.h | 1 +
 src/sv_main.cpp | 9 ++++---
 src/sv_main.h | 3 +-
 src/sv_master.cpp | 39 +++++++++++++++++++++++++++++++-----
 src/w_wad.cpp | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 src/w_wad.h | 7 +++++-
 src/win32/serverconsole/serverconsole.cpp | 7 +++--
 src/win32/serverconsole/serverconsole_settings.cpp | 9 ++++---
 16 files changed, 251 insertions(+), 54 deletions(-)