MantisBT - Doomseeker
View Issue Details
0003545Doomseeker[All Projects] Bugpublic2018-10-08 04:002018-10-27 22:53
Blzut3 
Zalewa 
highmajoralways
closedfixed 
1.1 
1.21.2 
0003545: Bzip2 IO device doesn't handle files with multiple streams
Apparently a bz2 file can contain multiple streams concatenated together. Wadseeker needs to reinitialize the decompressor at each boundary. Presumably this fact is utilized by programs that do multithreaded compression.
Have Wadseeker extract this Zandronum testing release:'https://zandronum.com/downloads/testing/3.1/ZandroDev3.1-180901-1533linux-x86_64.tar.bz2 [^]'

Release version of Wadseeker will silently fail to extract the binary.
No tags attached.
Issue History
2018-10-08 04:00Blzut3New Issue
2018-10-08 04:00Blzut3Statusnew => assigned
2018-10-08 04:00Blzut3Assigned To => Blzut3
2018-10-08 04:16Blzut3Note Added: 0019997
2018-10-08 04:16Blzut3Statusassigned => needs testing
2018-10-08 04:17Blzut3Fixed in Version => 1.2
2018-10-08 08:14WubTheCaptainNote Added: 0020001
2018-10-08 10:04WubTheCaptainNote Added: 0020007
2018-10-08 10:04WubTheCaptainAssigned ToBlzut3 => Zalewa
2018-10-08 10:04WubTheCaptainStatusneeds testing => needs review
2018-10-08 20:51ZalewaNote Added: 0020027
2018-10-08 20:52ZalewaNote Edited: 0020027bug_revision_view_page.php?bugnote_id=20027#r12199
2018-10-11 17:48ZalewaNote Added: 0020077
2018-10-11 17:58WubTheCaptainStatusneeds review => resolved
2018-10-11 17:58WubTheCaptainResolutionopen => fixed
2018-10-11 18:02WubTheCaptainNote Added: 0020078
2018-10-27 22:53WubTheCaptainStatusresolved => closed

Notes
(0019997)
Blzut3   
2018-10-08 04:16   
Forgot to tag ticket number in commit message:'https://bitbucket.org/Doomseeker/doomseeker/commits/9d64a42b9955e80c819c7da7f9cbb080ffb25866 [^]'

This took me way longer than I want to admit to figure out. That said I can't find much documentation on the bz2 file format so I don't feel too bad about it. Somehow though all the archiving tools know about it, so at least I can confirm that multiple implementations (including most importantly the reference bzip2 program) do this by tracing through the source codes.
(0020001)
WubTheCaptain   
2018-10-08 08:14   
I'm not favoring the "multiple nested ifs" and "if-else" coding style, personally.
(0020007)
WubTheCaptain   
2018-10-08 10:04   
Zalewa, please review/test?
(0020027)
Zalewa   
2018-10-08 20:51   
(edited on: 2018-10-08 20:52)
Quote from WubTheCaptain

I'm not favoring the "multiple nested ifs" and "if-else" coding style, personally.


The `ZandronumClientExeFile::downloadTestingBinaries()` method does a lot of stuff if anything. I'd start the refactor from splitting it into smaller ones, but that should be done on occasion of some more important change as there are no major issues with it requiring dedicated or immediate work.

Testing will have to wait until I'm less tired. I also don't have any bz2 archives readily available for testing.

(0020077)
Zalewa   
2018-10-11 17:48   
I packed dwango5.wad into a 'tar.bz2' archive using 3 different methods and added 'tar.bz2' to the supported WAD archive extensions in Wadseeker just for testing purposes. I have then attempted to install each archive with a Doomseeker build from before the patch and then with a build with the patch.

The commands and WAD installation results are as listed:

Before patch:

tar -cjf dwango5.tar.bz2 dwango5.wad # -- OK
tar -cf dwango5.tar dwango5.wad && bzip2 dwango5.tar # -- OK
tar -cf dwango5.tar dwango5.wad && pbzip2 -p16 -m1 -b1 dwango5.tar # -- FAIL

After patch:

tar -cjf dwango5.tar.bz2 dwango5.wad # -- OK
tar -cf dwango5.tar dwango5.wad && bzip2 dwango5.tar # -- OK
tar -cf dwango5.tar dwango5.wad && pbzip2 -p16 -m1 -b1 dwango5.tar # -- OK


The files created by `tar` and `bzip2` are the same while `pbzip2` creates a file with a different size.

So I suppose this proves that the problem is fixed.
(0020078)
WubTheCaptain   
2018-10-11 18:02   
I'll hope you'll get into a habit of using pipes instead of temporary files and &&, tee hee.