Bug 8402 - generate 'install_misc/debian.packages' file from debian/control file
Summary: generate 'install_misc/debian.packages' file from debian/control file
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Installation and upgrade (command-line installer) (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Mason James
QA Contact: Jonathan Druart
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-07-10 06:43 UTC by Mason James
Modified: 2020-01-06 20:14 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
bug 8402 - a script to build ./install_misc/debian.packages from ./debian/control (3.36 KB, patch)
2013-03-26 20:10 UTC, Mason James
Details | Diff | Splinter Review
bug 8402 - a script to build ./install_misc/debian.packages from ./debian/control (3.35 KB, patch)
2013-03-26 20:17 UTC, Mason James
Details | Diff | Splinter Review
bug 8402 - a script to build ./install_misc/debian.packages from ./debian/control (3.38 KB, patch)
2013-04-12 14:39 UTC, Kyle M Hall
Details | Diff | Splinter Review
bug 8402 - generate 'install_misc/debian.packages' file from debian/control file (3.30 KB, patch)
2013-04-25 11:57 UTC, Mason James
Details | Diff | Splinter Review
bug 8402 - generate 'install_misc/debian.packages' file from debian/control file (3.29 KB, patch)
2013-04-25 12:06 UTC, Mason James
Details | Diff | Splinter Review
bug 8402 - generate 'install_misc/debian.packages' file from debian/control file (3.23 KB, patch)
2014-11-13 04:28 UTC, Mark Tompsett
Details | Diff | Splinter Review
bug 8402 - generate 'install_misc/debian.packages' file from debian/control file (3.33 KB, patch)
2014-11-13 04:32 UTC, Mark Tompsett
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Mason James 2012-07-10 06:43:40 UTC
we now have 2 files in the Koha repo with debian package info

- install_misc/debian.packages
- debian/control

thats very bad!

deleting 'install_misc/debian.packages' would mean we only have one :)


the general plan is to delete the install_misc/debian.packages file from the repo

then dynamically-generate(tm) an 'install_misc/debian.packages' file using a script and the debian/control file, when installing Koha


something like this...

$ ./koha-debian-deps.pl > install_misc/debian.packages
$ sudo aptitude  $(<install_misc/debian.packages)
Comment 1 Mark Tompsett 2012-09-18 10:51:48 UTC
What populates C4/Installer/PerlDependencies.pm?

The part that isn't in PerlDependencies.pm are the various applications/commands that need installing as well, which I see are in other sections.

I agree there needs to be consistency, but what about the bigger picture of the perl dependencies required for a git install?

I noticed you did a "sudo aptitude" command-line. Is this preferred over a:
sudo dpkg --set-selections < install_misc/{OS}.packages
sudo dselect
way of doing it? aptitude may not be installed by default, and yes, neither is dselect.

Also, notice I said {OS}.packages? The library names may be different between debian-based OSes, and this is why I like "apt-file" for finding a reasonable name. Check out bug 8517 for a script I wrote which uses koha_perl_deps.pl to add missing items to {OS}.packages. I believe a newer incarnation of it is in the release-tools git repo.

As long as apt-file is installed and up to date and PerlDependencies is correct. It can be used to determine missing pieces and add them into the .packages file.

Does PerlDependencies.pm get updated more accurately than debian/control? Perhaps the solution isn't to delete debian/control which is likely used to build packages where as install_misc/debian.packages is used for a git install, but rather make sure they are both updated from what may be the source: PerlDependencies.pm?

Just some related brainstorming. :)
Comment 2 Mark Tompsett 2012-09-30 15:51:39 UTC
While thinking over the problem of Test::MockModule and DBD::Mock not being installed, I was chatting with "eythian" on the IRC channel. It turns out that that debian/update-control scripts uses the list-deps script which works it magic back to PerlDependencies.pm

$ cd debian
$ grep ": lib" control | wc -l
1
$ grep "^ lib" control | wc -l
85
$ cd ../install_misc
$ grep "^\s*lib" debian.packages | wc -l
102
$ cd ../C4/Installer
$ less PerlDependencies.pm
# I visually inspected to make sure everything was a tuple
$ grep "min_ver" PerlDependencies.pm | wc -l
106

Now here's one possible reason for the difference: two libraries (blah::foo::bar and blah::foo) are in the same package (libblah-foo-perl).

Useful commands that came to mind:
$ grep "^    '" PerlDependencies.pm | cut -f2 -d"'" | sort > ~/PerlDepsPerl.txt
$ grep "^    '" PerlDependencies.pm | cut -f2 -d"'" | sort | sed -e "s#::#-#g" | tr "[:upper:]" "[:lower:]" | perl -e "@data=<STDIN>; for \$line (@data) { chomp \$line; print \"lib\$line-perl\\n\"; }" > ~/PerlDepsLib.txt
$ grep "^    '" PerlDependencies.pm | cut -f2 -d"'" | sort | sed -e "s#::#\/#g" | perl -e "@data=<STDIN>; for \$line (@data) { chomp \$line; print \"\$line.pm\\n\"; }" > ~/PerlDepsPath.txt

$ cd
$ for i in `cat PerlDepsPath.txt`; do echo $i; apt-file search -x "^/usr/((lib|share)/perl5|(lib|share)/perl/[0-9.]+)/$i\$"; done

My thoughts are that install_misc/debian.packages could easily be built with a little bit of extra scripting, when the debian/control file is generated.

Though, I still need to understand how the |'s are handled in the control file, as they are listed as two lines in the sample for loop above.

This beginning to wade into the bowels of debian packaging stuffs also may have demonstrated that I need to tweak that check_deps.sh script some -- choose the non-xpm version of GD.
Comment 3 Mason James 2013-03-26 20:10:51 UTC Comment hidden (obsolete)
Comment 4 Mason James 2013-03-26 20:17:16 UTC Comment hidden (obsolete)
Comment 5 Kyle M Hall 2013-04-12 14:39:57 UTC Comment hidden (obsolete)
Comment 6 Mason James 2013-04-25 11:57:05 UTC Comment hidden (obsolete)
Comment 7 Mason James 2013-04-25 12:00:13 UTC
> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
> Works as advertised

i've made some small improvements to the script...
perhaps these changes should be re-tested?

i've set the patch back to 'needs signoff'
Comment 8 Mason James 2013-04-25 12:06:04 UTC Comment hidden (obsolete)
Comment 9 Jonathan Druart 2013-06-19 10:19:50 UTC
QA comment:

4 smalls remarks:
1/ FAIL	install_misc/koha-debian-deps.pl
   FAIL	  pod
   *** ERROR: Spurious text after =pod  in file install_misc/koha-debian-deps.pl
Need a blank line under "=pod"

2/ Please use Pod::Usage.

3/ Maybe should you use
   C4::Context->config('intranetdir') . '/debian/control'
instead of using a relative path.
I don't know if it is relevant given that is adds a dependence to C4::Context.

4/ It is useless to return 1 at the end of a perl script.

Marked as Failed QA.
Comment 10 Mark Tompsett 2013-07-11 17:50:27 UTC
Given that there is the ability to:
$ sudo apt-get install koha-deps koha-perldeps

Are
- install_misc/debian.packages
- install_misc/ubuntu.packages
- install_misc/ubuntu.12.04.packages
- install_misc/ubuntu.10.04.packages
- install_misc/ubuntu-packages.sh
- this script suggestion
even necessary?
Comment 11 Mason James 2013-07-18 03:54:59 UTC
(In reply to M. Tompsett from comment #10)
> Given that there is the ability to:
> $ sudo apt-get install koha-deps koha-perldeps
> 
> Are
> - install_misc/debian.packages
> - install_misc/ubuntu.packages
> - install_misc/ubuntu.12.04.packages
> - install_misc/ubuntu.10.04.packages
> - install_misc/ubuntu-packages.sh
> - this script suggestion
> even necessary?

no, i don't think they are necessary

i would be happy with a patch that removed these files
Comment 12 Mark Tompsett 2014-11-13 04:28:42 UTC Comment hidden (obsolete)
Comment 13 Mark Tompsett 2014-11-13 04:32:17 UTC
Created attachment 33507 [details] [review]
bug 8402 - generate 'install_misc/debian.packages' file from debian/control file

to test...

- rename ./install_misc/debian.packages file
    mv ./install_misc/debian.packages ./install_misc/debian.packages.old

- apply patch
    git bz apply 8402

- run script
   ./koha-debian-deps.pl -r > install_misc/debian.packages

- compare improved package file
    diff  ./install_misc/debian.packages ./install_misc/debian.packages.old

- install packages
    sudo apt-get install  $( < install_misc/debian.packages)

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Works as advertised
Comment 14 Mark Tompsett 2014-11-13 04:36:19 UTC
(In reply to Jonathan Druart from comment #9)
> QA comment:
> 
> 4 smalls remarks:
> 1/ FAIL	install_misc/koha-debian-deps.pl
>    FAIL	  pod
>    *** ERROR: Spurious text after =pod  in file
> install_misc/koha-debian-deps.pl
> Need a blank line under "=pod"

Added blank line.

 
> 2/ Please use Pod::Usage.

Not sure how to do that, but it's likely related to the print_usage() function.


 
> 3/ Maybe should you use
>    C4::Context->config('intranetdir') . '/debian/control'
> instead of using a relative path.
> I don't know if it is relevant given that is adds a dependence to
> C4::Context.

Which is good, because then:
./install_misc/koha-debian-deps.pl -r
works nicely without having to be in a fixed location.

 
> 4/ It is useless to return 1 at the end of a perl script.

removed.

Also, I probably should have cleaned up the licensing. ;)
I was also disappointed, because the list generated does not include gcc, which is required for some of the makes. So, I think I'll leave this Failed QA.
Comment 15 Josef Moravec 2019-03-13 08:59:46 UTC
install_misc directory was removed, so I think this is not valid anymore