0003552: DogSoft, included in Wadseeker's WAD sites, often gives outdated WADs (and/or is very slow)

Notes
(0020089) Blzut3 (administrator) 2018-10-12 23:14 edited on: 2018-10-12 23:21	Last I checked (admittedly a long time ago) mega-dog was really responsive. Wouldn't it be better to report bad files? I mean we're getting really close to "just don't even bother trying to find wads." :P Edit: Made an attempt to reach Mega-Dog for comment.

(0020090) WubTheCaptain (reporter) 2018-10-13 00:07	I'm not sure if this is temporary or a long standing issue, but'https://doom.dogsoft.net/ [^]' is also returning lots of 500 Internal Server Error status codes (about 50% of the time).

(0020095) Zalewa (developer) 2018-10-13 14:39 edited on: 2018-10-13 14:40	Last time I talked with Collin (mega-dog) was in November 2016. We discussed how Wadseeker operates so that he could adjust the server settings to minimize unnecessary resources use. Note that Wadseeker doesn't handle URLs that respond with HTTP attachments very well. When an archive or a .wad attachment is encountered, there are actually two downloads started with the first download being aborted after reading the HTTP meta-data. Any payload that already arrived in this first connection is discarded. Only the second connection actually gets the data. I explained all of this to him. Ever since that 2016 talk I noticed that the server was reconfigured to throttle the upload speed (from server's perspective) for a few initial seconds. This was presumably done so in expectance that the connection may be dropped after the client reads the meta-data. This throttling is understandable. I don't know what's going on with this service now. Quote from WubTheCaptain gives old wads which won't work. I also don't know what's going on with that. 10 years ago it was a common practice that if you planned on creating a continuously-developed WAD, you needed to include the version of that WAD in its filename. Did people abandon this practice today? When WAD authors don't follow this rule, it's their own fault that people are getting incompatible versions of WADs.

(0020136) Mega-Dog (reporter) 2018-10-19 12:21	I know about the internal server error, I am going to move over to a dedicated server probably in January since my host can't do anything about the overload on traffic at times. The throttle speed is due to the overload and I have no control over it and no matter how much I complain about everything to my host they ignore me and tell me something different like it is in my code which is not. As far as outdated wads, well if a real wad author would know to do different version numbers on their wads so it can always get the updated file. Even an author making a new wad would look to see if a wad already has a name. Majority of all the new files are archived off all the main servers in use, that is how i have more than 90% of the wads/pk3's these days. I can't believe that some people that are newer to the Doom community don't understand the effort that some people like me have put into gathering all the wad projects for over 10 years now. Bad enough I get emails of people even wanting a copy of my archive which I will probably never provide.

(0020137) WubTheCaptain (reporter) 2018-10-19 20:50	Mega-Dog, thanks for the response. Quote from Mega-Dog I know about the internal server error, I am going to move over to a dedicated server probably in January That's far ahead, but I wonder why not sooner since we both seem to be so unsatisfied in the current situation with the service provider quality. The site is currently inaccessible (HTTP status code 500) and probably has some impact on the UX of Wadseeker with default configuration. I don't know yet when a Doomseeker/Wadseeker 1.3 release would be. Quote from Mega-Dog tell me something different like it is in my code which is not. Side-note: Is the PHP code on the site available as free software for review? I bet it's quite trivial, but I guess it still uses trivially more host resources (CPU) for going through getwad.php instead of a static file on the server. I recognize getwad.php does some (anonymized?) statistics with downloads, I guess. Quote from Mega-Dog I can't believe that some people that are newer to the Doom community don't understand the effort that some people like me have put into gathering all the wad projects for over 10 years now. Bad enough I get emails of people even wanting a copy of my archive which I will probably never provide. I appreciate it. Side-note: I remember ArchiveTeam crawled and grabbed a large WAD site or two a while ago with ArchiveBot. There's interest for that, video game history preservation at the Internet Archive. This may or may not have been partly due to me; I've seen enough websites/filesharing sites come and go, some old some new.

(0020139) Blzut3 (administrator) 2018-10-20 00:37	Unfortunately "mod authors should," while true that it would avoid the problem, doesn't really solve the problem at hand. Plus given that you say you scan servers to archive their files it sounds like a malicious actor could potentially preemptively poison your archive. Do you have a preferred procedure that someone who notices and outdated mod can report it and have it updated? Also might be interesting now that Zandronum is going to have MD5 hash reporting (and Odamex had it for awhile) to support MD5 lookup.

(0020262) WubTheCaptain (reporter) 2018-12-17 04:43	No consensus, check again in January (per Mega-Dog).

(0020312) WubTheCaptain (reporter) 2019-01-16 15:26	Still slow and occassionally 500 Internal Server Error. Hosted on 67.210.124.50 (draco.lunarpages.com).

(0020419) WubTheCaptain (reporter) 2019-02-20 01:01	Now hosted on 69.195.128.234 (Joe's Datacenter, LLC) and it feels much snappier. I got full 100 Mbit/s download speeds with this. MegaDog delivered as promised, thanks.

This issue is already marked as resolved. If you feel that is not the case, please reopen it and explain why.
Supporters:	DrinkyBird WubTheCaptain
Opponents:	No one explicitly opposes this issue yet.

Issue History
Date Modified	Username	Field	Change
2018-10-12 21:56	WubTheCaptain	New Issue
2018-10-12 21:56	WubTheCaptain	Status	new => assigned
2018-10-12 21:56	WubTheCaptain	Assigned To	=> WubTheCaptain
2018-10-12 21:56	WubTheCaptain	File Added: 2018-10-12-215344_578x138_scrot.png
2018-10-12 22:08	WubTheCaptain	File Added: 0001-WAD-sites-Remove-Dogsoft.patch
2018-10-12 22:08	WubTheCaptain	Status	assigned => needs testing
2018-10-12 22:08	WubTheCaptain	Status	needs testing => needs review
2018-10-12 23:14	Blzut3	Note Added: 0020089
2018-10-12 23:21	Blzut3	Note Edited: 0020089	View Revisions
2018-10-13 00:07	WubTheCaptain	Note Added: 0020090
2018-10-13 01:29	WubTheCaptain	Summary	Dogsoft, included in Wadseeker's WAD sites, often gives outdated WADs (and/or is very slow) => DogSoft, included in Wadseeker's WAD sites, often gives outdated WADs (and/or is very slow)
2018-10-13 14:39	Zalewa	Note Added: 0020095
2018-10-13 14:40	Zalewa	Note Edited: 0020095	View Revisions
2018-10-13 16:33	WubTheCaptain	Target Version	1.2 => 1.3
2018-10-19 12:21	Mega-Dog	Note Added: 0020136
2018-10-19 20:50	WubTheCaptain	Note Added: 0020137
2018-10-20 00:37	Blzut3	Note Added: 0020139
2018-10-20 03:12	Blzut3	Relationship added	related to 0003369
2018-12-04 00:09	WubTheCaptain	Category	Bug => Wadseeker sites
2018-12-17 04:43	WubTheCaptain	Note Added: 0020262
2018-12-17 04:43	WubTheCaptain	Status	needs review => resolved
2018-12-17 04:43	WubTheCaptain	Resolution	open => suspended
2019-01-16 15:26	WubTheCaptain	Note Added: 0020312
2019-01-16 15:26	WubTheCaptain	Status	resolved => confirmed
2019-01-16 15:26	WubTheCaptain	Resolution	suspended => reopened
2019-02-20 01:01	WubTheCaptain	Note Added: 0020419
2019-02-20 01:01	WubTheCaptain	Status	confirmed => resolved
2019-02-20 01:01	WubTheCaptain	Resolution	reopened => no change required
2019-02-20 01:01	WubTheCaptain	Resolution	no change required => won't fix
2019-07-30 10:15	WubTheCaptain	Status	resolved => closed