Bug 8517

Summary: Script to automatically generate package lists
Product: Koha Reporter: Mark Tompsett <mtompset>
Component: contribs.koha-community.orgAssignee: Bugs List <koha-bugs>
Status: CLOSED FIXED QA Contact:
Severity: enhancement    
Priority: P3    
Version: unspecified   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Beta Version Script to find: things missing in repo, find matching in repo, find missing in {os}.{version}.packages files

Description Mark Tompsett 2012-07-27 17:27:37 UTC
Inspired by the mess cleaned up by bug 8489, I proceeded to develop a script that would:
- list packages not found in the default repository
- list packages found in the default repository
- list packages not found in the ubuntu.{version}.packages file list

Conversations with Robin Sheat gave me an useful grep expression to help with the hunt when combined with apt-file. I was trying to do it a harder way, when apt-file is perfect for this problem.

Conversations with Jared Camins-Esakov gave me the idea to expand the functionality beyond just Ubuntu, as apt-file is capable of using a specific sources.list file, rather than the default repositorys of one's OS.

The current help triggered on bad parameters reads something like:
===============================================================================
Usage: {path/to/}check_deps.sh [--koha-dir=<path>] [--list-dir=<path>]
                     [--dist=<dist>] [--release=<release>]
                     [--file=<file>] [--full=<full>]

koha-dir: The directory in which koha_perl_deps.pl is located.
          the default is `dirname "{path/to/}check_deps.sh"`.
list-dir: If you wish to use a specific source.list file
          you will need to specify the directory it is in.
    dist: This serves a dual role. Firstly, it is used to calculate
          the name of the source.list file, but also to determine
          which .packages file should be compared against in the
          install_misc directory under the koha directory.
          This should be specified along with the release parameter.
 release: This is the version number of the distribution of
          linux being used. It serves the same dual purpose.
          Sample valid values: 10.04, 12.04, 6.0.5
          The Debian value of 6.0.5 will be truncated to 6.0 solely.
          This should be specified along with the dist parameter.
    file: If you wish to keep the output around after running the
          script, give a file name to store it in. There will be
          three suffixes appended:
          1) .all - This is all the libraries it did find.
          2) .fix - These are the libraries missing from the
                    .packages file.
          3) .missing - This is lists libraries not found at all.
                        This means they should be added to the
                        koha-community repositories.
                        It's worthy of a bug report.
    full: Can be 0 or 1. For verbose output use 1. To just list
          missing files (0), that is the type 3 from above, are
          listed to STDOUT.

sample output line for found library:
   Net::Z3950::ZOOM ~ libnet-z3950-zoom-perl: /usr/lib/perl5/Net/Z3950/ZOOM.pm

sample output line for library that doesn't exist:
   {library name} NOT FOUND!

In the bizarre case that there is more than one possibility:
   GD ~: libgd-gd2-noxpm-perl: /usr/lib/perl5/GD.pm
   ~: libgd-gd2-perl: /usr/lib/perl5/GD.pm
GD is currently the only case, and so has a special exception.
If you find a new special case, please bug report it at
http://bugs.koha-community.org/bugzilla3

Sample uses: check_deps.sh | grep 'NOT FOUND'
             Find libraries which aren't in any repository.
             If you see {library name}, the parameters are wrong.

Sample uses: check_deps.sh | grep '~' | cut -f2- -d'~' | cut -f1 -d':'
             Find libraries which can be 'apt-get install'd
===============================================================================

Output of the script may need some tweaking. However, given that the three files are generated, it could be completely ignored too. Though, I find I like to read it just to be sure.

P.S. Try not to hurt my baby.
Comment 1 Mark Tompsett 2012-07-27 17:35:22 UTC
Created attachment 11175 [details]
Beta Version Script to find: things missing in repo, find matching in repo, find missing in {os}.{version}.packages files

This should work on any debian-based OS. I was doing my testing under Ubuntu.
I hope there are enough comments in the script that it is easy enough to fix if there are problems.

I call it Beta, because obviously one tester currently (myself) does not make for a perfect product.

Jared, could you patch this up for me? I am uncertain which repository you were thinking of placing this under, and I probably don't have a git clone of it.
Comment 2 Mark Tompsett 2012-07-28 06:50:34 UTC
I started tinkering with CentOS again. GAH! apt-file is a big plus for debian-based systems. Though, I did discover repoquery -q -l.

CentOS 5.x does not come with Perl 5.10 by default. CentOS 6.x does, as far as I know. So in terms of even attempting Koha installs on CentOS, 6.x is the only real option for 3.6 and up (which are the currently active Koha versions).

I'm putting some of my notes here in hopes of extending this script across to the RPM-based world. However, at this point, it is looking slim (at least with CentOS):
http://paperlined.org/rosetta_stone/package_systems.html
http://lists.centos.org/pipermail/centos/2008-September/065192.html

Deb-based OS' track perl development better the RPM-based ones, it would seem.

http://wiki.koha-community.org/wiki/Koha_3.6_on_Centos_6.2_i386
What bother's me most is that there are manual fixing stages to these instructions. I am hoping to figure out a way that doesn't need that. But I make no guarantees.

I also note that the EPEL repository has been known to break things. That's part of the reason why there is the need to fix things. Whereas, RPMForge "is considered by many in the community to be stable and safe." (http://wiki.centos.org/AdditionalResources/Repositories)