MantisBT - Doomseeker
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0003808Doomseeker[All Projects] Cleanuppublic2020-06-07 03:352020-06-09 08:12
ReporterWubTheCaptain 
Assigned To 
PrioritynoneSeverityminorReproducibilityalways
StatusacknowledgedResolutionopen 
PlatformOSOS Version
Product Version1.3.1 
Target VersionFixed in Version 
Summary0003808: Third-party dependencies are (bothersome) under src/ directory
DescriptionIt annoys me that some dependencies like LZMA or qt-json are under src/ directory. I can't search reasonably with grep -r to find out how many lines of code have issues in our code, it's a bit bothersome to exclude them (so I don't typically exclude any directories or files). Many issues (lines of code) grep finds (piped to wc -l) could be in those third-party libraries, which I'm not going to patch or ever touch.
I'd like those third-party dependencies to be in some other dependency directory.
Steps To ReproduceAn example is from issue 0003807's steps to reproduce being slightly inaccurate with numbers.

They are, for example, src/wadseeker/lzma/ and src/core/json.{cpp,h}.
Additional Information
TagsNo tags attached.
Relationships
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2020-06-07 03:35WubTheCaptainNew Issue
2020-06-07 03:36WubTheCaptainDescription Updatedbug_revision_view_page.php?rev_id=13123#r13123
2020-06-07 03:38WubTheCaptainNote Added: 0021361
2020-06-07 03:39WubTheCaptainSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=13125#r13125
2020-06-07 03:40WubTheCaptainRelationship addedrelated to 0003795
2020-06-07 03:41WubTheCaptainRelationship deletedrelated to 0003795
2020-06-07 19:55Pol MNote Added: 0021380
2020-06-07 19:55Pol MStatusnew => acknowledged
2020-06-08 01:01Blzut3Note Added: 0021403
2020-06-08 01:38Pol MNote Added: 0021407
2020-06-08 13:30ZalewaNote Added: 0021434
2020-06-08 13:31ZalewaNote Edited: 0021434bug_revision_view_page.php?bugnote_id=21434#r13186
2020-06-08 14:04WubTheCaptainNote Added: 0021439
2020-06-08 14:08ZalewaNote Edited: 0021434bug_revision_view_page.php?bugnote_id=21434#r13188
2020-06-08 14:43WubTheCaptainNote Added: 0021445
2020-06-08 21:34Blzut3Note Added: 0021452
2020-06-09 05:06WubTheCaptainNote Added: 0021453
2020-06-09 08:12ZalewaNote Added: 0021454

Notes
(0021361)
WubTheCaptain   
2020-06-07 03:38   
0003310:0018762:
Quote from Zalewa
These files are necessary to build Wadseeker and Doomseeker. If these files were to be moved out of Wadseeker source directory, building a valid Wadseeker source tarball would be that much harder.
(0021380)
Pol M   
2020-06-07 19:55   
Moving them out into a separate directory and building them into each target should not cause any problem (I think). The code will be duplicated into each binary, no actual change should happen.
(0021403)
Blzut3   
2020-06-08 01:01   
Open to proposals for layout changes, but yes the constraint would be that whatever directory the wadseeker project is in needs to be self contained. Similarly plugin specific code should be contained within a directory.
(0021407)
Pol M   
2020-06-08 01:38   
Quote

whatever directory the wadseeker project is in needs to be self contained

Oh, I forgot that.
(0021434)
Zalewa   
2020-06-08 13:30   
(edited on: 2020-06-08 14:08)
Convenient provisioning of external dependencies can be (with larger or lesser effort) solved with CMake's ExternalProject, thus allowing to remove all 3rd party open-source packages from the repo. Doing this would require changes to how Doomseeker/Wadseeker are configured and built: CMakeLists, documentation, people habits, the source packager scripts. In corner cases unforgivable bugs such as 0003759 can ensue.

It could be easier to move src/wadseeker/lzma to the dependencies/ topdir. Given that we already have the makesourcepackages.sh script, this script can be used to produce the self-contained Wadseeker tarball. Thanks to WSL I can also run shell scripts now without any inconvenience (but that's a digression).

But... this movement would be useful if there was active (and I mean *ACTIVE*) development going on around here. With how much stuff isn't happening, this is moving things around just to move them.

I'm inclined to leave this issue be for the moment and revisit it at a more favorable time.
(0021439)
WubTheCaptain   
2020-06-08 14:04   
Quote from Zalewa
In corner cases unforgivable bugs such as 0003579 can ensue.

Typo, 0003759?
Quote from Zalewa
I'm inclined to leave this issue be for the moment and revisit it at a more favorable time.

There is no priority on this issue, all things adjusted eventually.
(0021445)
WubTheCaptain   
2020-06-08 14:43   
Quote from Zalewa
But... this movement would be useful if there was active (and I mean *ACTIVE*) development going on around here. With how much stuff isn't happening, this is moving things around just to move them.


Doomseeker is quite feature complete (since 1.1 or 1.2 I'd say; 1.3 if you want to count WAD hash comparison). It's bugfixes and maintenance at this point.
(0021452)
Blzut3   
2020-06-08 21:34   
I seem to recall Kitware saying somewhere that ExternalProject isn't necessarily supposed to be used except as a last resort. Or as a way to drive an overarching mega build project. Not sure where I saw that though.
(0021453)
WubTheCaptain   
2020-06-09 05:06   
Quote from Blzut3
I seem to recall Kitware saying somewhere that ExternalProject isn't necessarily supposed to be used except as a last resort. Or as a way to drive an overarching mega build project. Not sure where I saw that though.

'https://blog.kitware.com/cmake-superbuilds-git-submodules/ [^]'
(0021454)
Zalewa   
2020-06-09 08:12   
This article seems to have a rather positive outlook on ExternalProject.

The use case for ExternalProject I had in mind was something I've tested for our other project and recently also at work. It has worked well for me. It is meant to solve the unavailability of pre-packaged binaries or even pre-configured build scripts for 3rd party open-source libs on platforms other than Linux.

To put it simple:

1. The main project (ie. Doomseeker, Wadseeker) doesn't use ExternalProject anywhere in the CMakeLists.txt. It keeps using find_package() as usual. This way you can still use the system's or preinstalled dependencies when it's more appropriate or desireable.

2. There is another dependency project, separate from the main project, that is supposed to download, build and install all of the ExternalProjects. The programmer needs to build and install this project first, then the main project is configured with CMAKE_PREFIX_PATH set to CMAKE_INSTALL_PATH of this dependency project.

Since the dependencies and the main project are now compiled from the source by the same compiler, you're now certain that all the libs have compatible ABI and runtime dependencies.

This makes the build two-step on non-Linux platforms but this is more acceptable and way more convenient than having to scrape all the dependencies manually off the net.

I believe this article describes this type of ExternalProject use as "Dependency build".