Zandronum Chat @ irc.zandronum.com
#zandronum
Get the latest version: 3.0
Source Code

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0003237Doomseeker[All Projects] Bugpublic2017-09-01 13:282017-09-18 18:31
ReporterWubTheCaptain 
Assigned To 
PrioritynormalSeverityblockReproducibilityN/A
StatusnewResolutionopen 
Platformx86_64 (really cross-platform)OSDebian GNU/LinuxOS Versionbuster/sid
Product Version1.1 
Target VersionFixed in Version 
Summary0003237: Non-free files and copyright issues in source archive
DescriptionTo create package suitable for inclusion in Debian GNU/Linux, much of Doomseeker's source code needs to be omitted, patched and/or repackaged without files which don't come with a copyright or license notice to meet Debian Free Software Guidelines (DFSG). There are also non-free files or files without copyright/license notice, which is a blocker for redistributing packages.

Arguably it's unsafe to assume everything to be under GPLv2(+) without explicitly stating so, and also impossible to tell if a source file is under "GPLv2 only" or "GPLv2+" (any later version) without a license heading. The origins of each file must be known certainly for inclusion.

Not every file needs to be under GPLv2+ of course, only under a compatible license. GNU All-Permissive license is suitable for small README files, for example. For media files and such where metadata is not possible to attach, human-readable instructions of licensing in COPYING file or similar will do the work.

One concrete example: tools/geolite2_conv.py has no free license stated, but claims "(c) The Doomseeker Team 2016." (Whether this is enforceable or not is a different issue, depending on if "The Doomseeker Team" is a legal entity recognized by courts.)

Other issues: In some non-Berne convention countries, (c) or (C) alone may not be enough and may need to include the word "Copyright" or symbol ©. Lintian tag copyright-with-old-dh-make-debian-copyright.
Steps To Reproduce

  1. wgethttp://doomseeker.drdteam.org/files/doomseeker-1.1_src.tar.bz2 [^] # Retrieves the source archive

  2. tar xvfz doomseeker-1.1_src.tar.bz2 # Extracts the archive

  3. Use standard system utilities (see intro(1) man page) to browse and read the files.

  4. licensecheck --recursive ./ reproduces attached licensecheck.log.

  5. debmake -c reproduces debmake-c.log (may need to create Debian stub files with dh_make first).



Additional InformationAttached logs also have false positives, but many true positives too.
Attached Fileslog file icon debmake-c.log [^] (172,856 bytes) 2017-09-01 13:28
log file icon dpkg.log [^] (506 bytes) 2017-09-01 13:28
log file icon licensecheck.log [^] (144,059 bytes) 2017-09-01 13:28
log file icon licensecheck.problems-and-ambiguities.log [^] (13,793 bytes) 2017-09-18 16:48

- Relationships
child of 0003246new Debian packaging. 

-  Notes
User avatar (0018208)
WubTheCaptain (reporter)
2017-09-01 13:31
edited on: 2017-09-01 13:31

In example, the dependencies folder comes with binary Windows objects for bzip2 1.0.6 and zlib 1.2.3. The Debian package maintainer needs to repack a DFSG source archive without these objects for each release (which can be somewhat automated in debian/copyright Files-Excluded field).

User avatar (0018214)
WubTheCaptain (reporter)
2017-09-01 16:32
edited on: 2017-09-01 16:33

I also found out there's no real documentation about Doomseeker's binaries being redistributable under GPLv3(+?) only, because of LGPLv3(+?) dependency on Qt5.

Quite the opposite, in fact: "This program is distributed under the terms of the GPL v2." in about dialog. Uh oh.

User avatar (0018232)
Blzut3 (administrator)
2017-09-06 01:15

Doomseeker is GPLv2+, Wadseeker is LGPLv2.1+ so while the about page is wrong it is in the clear. I have no issues with fixing this although any chance you might have a link to an explanation of what makes LGPLv2.1 incompatible with LGPLv3? All I can find without digging into the text of the license itself is the statement that it is not. Which seems strange to me if I could mix it with 100% proprietary code, but not GPLv2/LGPLv2.1. Unless it's just a quirk with how the GPL requires that all code fit under its exact terms.
User avatar (0018234)
WubTheCaptain (reporter)
2017-09-06 05:54
edited on: 2017-09-17 17:13

There might have been a miscommunication or misunderstanding. The Free Software Foundation has published a compatibility table:https://www.gnu.org/licenses/gpl-faq.en.html#AllCompatibility [^]

So if you want to use a library under LGPLv3 (Qt5), the combination (executable) should be ok under GPLv3. (No word on GPLv3+ and LGPLv3+, but I would assume the "or later" clause applies.)

Qt4 could be built with LGPL 2.1-mode and I tested I could still build Doomseeker 1.1 with Qt4. Thus Qt4 builds can remain under GPLv2+, while Qt5 are GPLv3(+).

User avatar (0018304)
Zalewa (developer)
2017-09-17 17:04

In regards to licenshecheck tool:

1. How do we satisfy the errors it prints for .png files or other images?

2. How do we satisfy the errors for code that is not ours?

2.1. Some of this belongs to bzip/zlib dependencies that can be statically linked into Doomseeker but also can be replaced by OS libraries: bzip/zlib. This code is actually forcefully statically linked in Windows builds because finding compatible, pre-built library versions on Windows is a nightmare born in hell.

2.2. There is some code that is used to build tools that are:

a) only needed during building and not distributed at all, or
b) distributed only on Windows as separate .exe files

2.3. lzma is "public domain", yet the tool spills out problems. Should we ignore that?

2.4. .ui and .ts files are generated by Qt tools.

2.5. doxygen.* files are generated by doxygen, albeit we can alter them pretty safely to add license header. But, should we? It was generated by a tool, even if it was later modified by hand.

2.6. ./src/core/CMakeFileListing.txt is a file that is generated by building tools.

3. tools/doomseeker-portable.bat is a one-liner. Do I need to add "rem" lines with GPL preamble there? This file is only needed on Windows, so probably not due to being out of scope for Debian.

4. What is the exact definition of done here? If I grep the licensecheck log file by UNKNOWN it spills out 1174 lines. Do we need to get this down to zero?


$ grep UNKNOWN licensecheck.log-1.txt | wc -l
1174


If I grep out files that I believe should not be touched by us, I get a smaller list where most files I can agree need amending, however, how can I be sure that whatever I'll do will be satisfactory?


$ grep UNKNOWN licensecheck.log-1.txt | \
    grep -i -v \\.png | \
    grep -v tools/updateinstaller | \
    grep -v tools/updaterevision | \
    grep -v /lzma/ | \
    grep -v \\.ts | \
    grep -v \\.ui | \
    grep -v /bzip2/ | \
    grep -v /zlib/ | \
    grep -v /dependencies/ | \
    wc -l
108
User avatar (0018305)
Zalewa (developer)
2017-09-17 17:28

Since the licensing in the source files indeed states that any "later" (L)GPL version can be used, we should probably amend all text that states that Doomseeker/Wadseeker is on (L)GPLv2(.1) and doesn't state "or later".
User avatar (0018314)
WubTheCaptain (reporter)
2017-09-18 08:49
edited on: 2017-09-18 13:43

Quote from Zalewa
1. How do we satisfy the errors it prints for .png files or other images?


Those can be manually reviewed by the packager. As long as there's a notice somewhere stating everything under X is licensed under Y (or a COPYING file in that subdirectory), that's good enough for me. Some may interpret more broadly everything not explicitly stated to be under the project's license (GPLv2+/COPYING). Of course, it must a free license or else it may be excluded or replaced in Debian.

Quote from Zalewa
2. How do we satisfy the errors for code that is not ours?


Can you give an example?

If it can be satisfied with my answer to #1, that's fine. If it's non-free, then it should be replaced or removed (or if distribution permits, at most non-free archive in Debian). If it's non-essential to a Debian package, then it may be excluded in a DFSG release.

Quote from Zalewa
2.1. Some of this belongs to bzip/zlib dependencies that can be statically linked into Doomseeker but also can be replaced by OS libraries: bzip/zlib. This code is actually forcefully statically linked in Windows builds because finding compatible, pre-built library versions on Windows is a nightmare born in hell.


I have proposed at issue 0003238 to distribute those seperately (and create documentation how to acquire/build it on Windows). I'm not sure what exactly would need to be done to do this.

bzip2 license is somewhat similar to a BSD-license/zlib license and permits binary distributions sure. zlib is the same and compatible with GPL, but I believe the problem expressed here is the conflict with Debian's policy. The binaries without source can be excluded from a DFSG package in Debian, probably.

Quote from Zalewa
2.2. There is some code that is used to build tools that are


If make clean can remove it after build, that's fine for Debian I suppose. Not sure what the question (if any) was here.

Quote from Zalewa
2.3. lzma is "public domain", yet the tool spills out problems. Should we ignore that?


The code seems to be public domain, so it's a false positive in licensecheck. I can handle this manually. The documentation/specifications licensing is a bit more ambiguous: src/wadseeker/lzma/7zC.txt for example has this top heading and license text:

7z ANSI-C Decoder 9.35
----------------------

...

LICENSE
-------

7z ANSI-C Decoder is part of the LZMA SDK.
LZMA SDK is written and placed in the public domain by Igor Pavlov.


I would assume good faith of the creative mind behind the work and assume all of it to be under public domain without warranty expressions. I doubt it causes any problems for Debian, but if it does the specifications can be excluded in a DFSG release but the code will stay.

Quote from Zalewa
.ui and .ts files are generated by Qt tools.


This is a false positive too. Qt is free, so there's no issue with freedom. .ts files seem to be translations created by this project, so they're covered under Doomseeker's license (GPLv2+). Probably the same for .ui files:

Quote from LGPLv2.1
The act of running a program using the Library is not restricted, and output from such a program is covered only if its contents constitute a work based on the Library (independent of the use of the Library in a tool for writing it).


A copyright notice (who authored the UI design/translations in what year) would be preferable to mention/document somewhere, although not a strict requirement. Mercurial commit logs may be enough for this.

Quote from Zalewa
2.5. doxygen.* files are generated by doxygen, albeit we can alter them pretty safely to add license header. But, should we? It was generated by a tool, even if it was later modified by hand.


See #1: An external COPYING notice is fine, a false positive in that case. Preferable if one can build the doxygen files from doxygen sources instead.

Quote from Zalewa
2.6. ./src/core/CMakeFileListing.txt is a file that is generated by building tools.


See previous.

Quote from Zalewa
tools/doomseeker-portable.bat is a one-liner.


A command to start the .exe executable with --portable flag is below threshold of originality, ineligible for copyright/public domain as stand-alone. False positive in such case.

Quote from Zalewa
What is the exact definition of done here? If I grep the licensecheck log file by UNKNOWN it spills out 1174 lines. Do we need to get this down to zero?


There's false positives, sure. Preferably do what you can for Doomseeker/Wadseeker sources, but it's not a strict requirement if the files are free. Doing so would at least make a package maintainer's life easier and avoid ambiguity for redistributors.

Autogenerated files probably don't need to be touched where not feasible.

Remove or replace whatever is non-free and can't be distributed under GPLv2+ from the main source code distribution.

User avatar (0018315)
WubTheCaptain (reporter)
2017-09-18 09:02
edited on: 2017-09-18 09:04

Quote from Zalewa
Since the licensing in the source files indeed states that any "later" (L)GPL version can be used, we should probably amend all text that states that Doomseeker/Wadseeker is on (L)GPLv2(.1) and doesn't state "or later".


Yes, I believe so. I believe else the non-source forms from build output will be distributable under GPLv2 or LGPLv2.1 only.

User avatar (0018322)
WubTheCaptain (reporter)
2017-09-18 14:13

I will look further into this.

Doxygen source is GPLv2 with Qt exception. Whether Doomseeker or Doxygen's copyright applies to Doxygen files I'm not entirely sure, but for distribution it doesn't matter because the license is all the same.

I'll post a cleaned up licensecheck log once finished to better define "done" for this ticket.
User avatar (0018323)
WubTheCaptain (reporter)
2017-09-18 17:01
edited on: 2017-09-18 17:04

licensecheck.problems-and-ambiguities.log is now available. An overview:


  • Some have real licenses or are ineligible for copyright, so I've removed them manually from the output.

  • Some things will be nuked from Debian DFSG package. Those are marked appropriately and should have consideration to be removed where possible.

  • media/icons/buddies.png seems to be non-free and should be replaced (unfortunately). Freedoom, anyone?

  • Exact sources for some media icons are missing. Some are ineligible for copyright. Flags are public domain.

  • Zalewa has some Doomseeker code in there with copyright notices but no permission for GPLv2+. Those are marked too.

  • media/slotstyles/marines/ I'm unsure, so I left it up for a question.

  • Core source has few files without copyright notice and no permission notice.

  • tools/updateinstaller/external/AnyOption/* is a non-free release, needs update to Expat released version.

  • tools/updateinstaller/external/verpatch/* is CPOL licensed, non-free. Must be removed.

  • tools/updateinstaller/external/win32cpp/* has more of this CPOL crap.

  • tools/updateinstaller/* in general is a pile of stinky poo I don't want to deal with. Code duplication (minizip), zlib contribs (removed in Debian), just a pain in the ass.

  • CMakeLists.txt files are probably copyrightable and manually crafted.



https://www.gnu.org/licenses/license-list.en.html#cpol [^]

User avatar (0018324)
Zalewa (developer)
2017-09-18 17:15

Thankfully, nothing from tools/updateinstaller/ is needed on Linux.
User avatar (0018325)
WubTheCaptain (reporter)
2017-09-18 17:18

True, but I believe tools/updateinstaller/* would need to be distributed seperately – if at all. How's Windows binaries, are they legal?
User avatar (0018326)
Zalewa (developer)
2017-09-18 18:31

Quote from "WubTheCaptain"
How's Windows binaries, are they legal?

Not in the clear, I would suspect, as one thing that is definitely missing is LICENSE file for the updater executable.

The update tool works like this:

1. As the "package maintainer", you build the complete software and `make install` it into a temporary directory. This prepares a full deployment of Doomseeker that you can now pack into a .zip file and redistribute to other computers. The program will run after extraction on any Windows XP or newer without any external dependencies needed - MSVC runtime DLLs are also provided in the package.

2. Mendeley Updater is also built and `make installed` as an "updater.exe" executable, which is bundled in Doomseeker package archive next to doomseeker.exe. We have altered Mendeley Updater's code to fit Doomseeker "theme", so: we changed the icon to Doomseeker's, the executable file metadata to state our ownership and also we changed some code because of problems. The customization alterations were done in accordance to the customization guidelines in Mendeley's README.md:https://github.com/Mendeley/Update-Installer#customizing-the-updater [^]

2.1. No part of Mendeley Updater is linked with doomseeker.exe.

3. Mendeley Updater comes with 2 Ruby scripts - one of those came with the original Mendeley Updater, but had to be modified by us due to problems. The other script was authored by us. These scripts are not `make installed` or otherwisely linked with any part of Doomseeker and only serve the "package maintainer" to build the update packages. When you run these Ruby scripts, showing them the directory where you `make installed` Doomseeker, they prepare .zip archives and .xml description files that is understood and handled by Mendeley Updater. Each .zip gets an .xml. How many are there depends on 4.

4. There are additional .js files that contain JSON structure understood by those Ruby scripts. They are located in "tools/updateinstaller/tools/(win32|macosx)-configs" directories. These .js files describe which files go into which update package. We build several separate packages: the core program, 3rd party dependency DLLs (Qt, MSVC), Wadseeker and each plugin also has its own update package. This is done to a) preserve bandwidth and b) allow to omit updating plugins that the user doesn't have in the first place. As with 3., these .js files are not packaged into Doomseeker.

5. Each package mentioned in 3. is also listed in a single, generated update-info.js file. This file describes the newest versions of each package, their names and download URLs.

6. Once we build the packages and the update-info.js file we upload them to a special directory on our website. From this point users will see that there's a new update available.


Now, the update process works like this:

1. Doomseeker (as in doomseeker.exe) downloads the update-info.js file from our website.

2. It compares package versions from this file to package versions loaded in its process's memory.

3. If there are any discrepancies (yes, even if the version number in the file is lower than the installed one), it will offer the download. When download completes, Doomseeker will notify the user that restart is required to install the updates. Depending on user configuration this process might be fully or half-automatic.

4. Upon shutdown or next start of "doomseeker.exe", it is detected that an update is pending installation and "updater.exe" is called with appropriate command line arguments. The "updater.exe" needs to be first copied by Doomseeker into a temporary file itself and run from that temporary copy, because Windows forbids overwriting of executables or DLLs that are running as processes. Running that temporary copy allows "updater.exe" to overwrite itself.

5. "updater.exe" overwrites the installed files with files from the update packages and restarts "doomseeker.exe" with appropriate arguments. It can notify "doomseeker.exe" if installation was successful or not through command line arguments.

Issue Community Support
Only registered users can voice their support. Click here to register, or here to log in.
Supporters: No one explicitly supports this issue yet.
Opponents: No one explicitly opposes this issue yet.

- Issue History
Date Modified Username Field Change
2017-09-01 13:28 WubTheCaptain New Issue
2017-09-01 13:28 WubTheCaptain File Added: debmake-c.log
2017-09-01 13:28 WubTheCaptain File Added: dpkg.log
2017-09-01 13:28 WubTheCaptain File Added: licensecheck.log
2017-09-01 13:31 WubTheCaptain Note Added: 0018208
2017-09-01 13:31 WubTheCaptain Note Edited: 0018208 View Revisions
2017-09-01 16:30 Zalewa Relationship added child of 0003246
2017-09-01 16:32 WubTheCaptain Note Added: 0018214
2017-09-01 16:33 WubTheCaptain Note Edited: 0018214 View Revisions
2017-09-06 01:15 Blzut3 Note Added: 0018232
2017-09-06 05:54 WubTheCaptain Note Added: 0018234
2017-09-17 17:04 Zalewa Note Added: 0018304
2017-09-17 17:13 Zalewa Note Edited: 0018234 View Revisions
2017-09-17 17:28 Zalewa Note Added: 0018305
2017-09-18 08:49 WubTheCaptain Note Added: 0018314
2017-09-18 08:55 WubTheCaptain Note Edited: 0018314 View Revisions
2017-09-18 08:56 WubTheCaptain Note Edited: 0018314 View Revisions
2017-09-18 09:02 WubTheCaptain Note Added: 0018315
2017-09-18 09:04 WubTheCaptain Note Edited: 0018315 View Revisions
2017-09-18 13:43 WubTheCaptain Note Edited: 0018314 View Revisions
2017-09-18 14:13 WubTheCaptain Note Added: 0018322
2017-09-18 16:48 WubTheCaptain File Added: licensecheck.problems-and-ambiguities.log
2017-09-18 17:01 WubTheCaptain Note Added: 0018323
2017-09-18 17:04 WubTheCaptain Note Edited: 0018323 View Revisions
2017-09-18 17:15 Zalewa Note Added: 0018324
2017-09-18 17:18 WubTheCaptain Note Added: 0018325
2017-09-18 18:31 Zalewa Note Added: 0018326






Questions or other issues? Contact Us.

Links


Copyright © 2000 - 2017 MantisBT Team
Powered by Mantis Bugtracker