Bug 25068

Summary: koha-common build error caused by missing /etc/searchengine
Product: Koha Reporter: Joonas Kylmälä <joonas.kylmala>
Component: PackagingAssignee: David Cook <dcook>
Status: Pushed to master --- QA Contact: Tomás Cohen Arazi <tomascohen>
Severity: blocker    
Priority: P1 - high CC: dcook, jonathan.druart, joy, martin.renvoize, mirko, mtj, tomascohen
Version: master   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24735
Change sponsored?: --- Patch complexity: Small patch
Who signed the patch off: Text to go in the release notes:
Version(s) released in:
20.05.00
Bug Depends on: 24735    
Bug Blocks: 25485, 25510, 25529    
Attachments: Bug 25068: Fix koha-common.install directives
Bug 25068: Fix koha-common.install directives
Bug 25068: Fix koha-common.install directives

Description Joonas Kylmälä 2020-04-06 07:24:36 UTC
Commit 52e95fff34e1c6da967a4d86fbb99bec748ccb31 from Bug 24735 removes the directory etc/searchengine which causes the debian build to fail:

> make[1]: Leaving directory '/build/koha-16.06~git+20200406100416.c800ef7e'
>    dh_install -O--fail-missing
> dh_install: Cannot find (any matches for) "debian/tmp/etc/koha/searchengine" (tried in "." and "debian/tmp")
> dh_install: koha-common missing files: debian/tmp/etc/koha/searchengine
> dh_install: missing files, aborting
> debian/rules:9: recipe for target 'binary' failed
> make: *** [binary] Error 2
> dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
> I: copying local configuration
> E: Failed autobuilding of package
Comment 1 Jonathan Druart 2020-04-09 11:31:51 UTC
I followed the steps from the wiki (https://wiki.koha-community.org/wiki/Building_Debian_Packages_-_The_Easy_Way), but I get

E: pbuilder-satisfydepends failed.
I: Copying back the cached apt archive contents
I: unmounting dev/ptmx filesystem
I: unmounting dev/pts filesystem
I: unmounting dev/shm filesystem
I: unmounting proc filesystem
I: unmounting sys filesystem
I: cleaning the build env 
I: removing directory /var/cache/pbuilder/build/25742 and its subdirectories
Command 'pdebuild  --buildresult /home/jonathan/debian' returns an error (256)

on `sudo ./debian/build-git-snapshot -r ~/debian -v 19.12.67git -d`.

I guess you could remove the following line to fix your issue:
debian/koha-common.install:debian/tmp/etc/koha/searchengine
Comment 2 Joonas Kylmälä 2020-04-09 11:34:59 UTC
The wiki instructions don't work since one needs to use for example stretch to build Koha instead of the default that is squeeze. And yes, removing that line from koha-common.install fixes the issue but I don't want to submit a patch for this until I know how we are supposed to configure Elasticsearch with the Debian package.
Comment 3 David Cook 2020-05-18 02:58:46 UTC
Yep, I'm getting the problem too when using https://gitlab.com/minusdavid/koha-deb-builder-docker and master:

make[1]: Leaving directory '/home/koha/build/koha-19.12.00-test+20200518020508.3dc1f8a8'
   dh_install -O--fail-missing
cp: cannot stat 'debian/tmp/debian/tmp/etc/koha/searchengine': No such file or directory
dh_install: cp -a debian/tmp/debian/tmp/etc/koha/searchengine debian/koha-common//etc/koha/ returned exit code 1
debian/rules:9: recipe for target 'binary' failed
make: *** [binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
debuild: fatal error at line 1376:
dpkg-buildpackage -rfakeroot -D -us -uc failed
Comment 4 David Cook 2020-05-18 03:00:35 UTC
(In reply to Joonas Kylmälä from comment #2)
> The wiki instructions don't work since one needs to use for example stretch
> to build Koha instead of the default that is squeeze. And yes, removing that
> line from koha-common.install fixes the issue but I don't want to submit a
> patch for this until I know how we are supposed to configure Elasticsearch
> with the Debian package.

I haven't used Elasticsearch with Koha yet, so could you speak more to that one? 

It looks like the Elastic search config lives in "admin/searchengine/elasticsearch/field_config.yaml" rather than "etc/searchengine"?
Comment 5 David Cook 2020-05-18 03:16:11 UTC
Created attachment 104993 [details] [review]
Bug 25068: Fix koha-common.install directives

Bug 24735 removed etc/searchengine/queryparser.yml, which removed
the need for koha-common.install to install the following file:
debian/tmp/etc/koha/searchengine

This patch removes that line and causes the Koha packages to build
correctly again.

To test:
0) Apply patch
1) Build a Koha package using one of the following methods:
- https://wiki.koha-community.org/wiki/Building_Debian_Packages_-_The_Easy_Way
- https://hub.docker.com/r/koha/koha-dpkg
- https://gitlab.com/minusdavid/koha-deb-builder-docker
Comment 6 David Cook 2020-05-18 05:55:15 UTC
Upping this one to critical as you can't build packages until it's fixed
Comment 7 Mason James 2020-05-18 10:59:04 UTC
Created attachment 105010 [details] [review]
Bug 25068: Fix koha-common.install directives

Bug 24735 removed etc/searchengine/queryparser.yml, which removed
the need for koha-common.install to install the following file:
debian/tmp/etc/koha/searchengine

This patch removes that line and causes the Koha packages to build
correctly again.

To test:
0) Apply patch
1) Build a Koha package using one of the following methods:
- https://wiki.koha-community.org/wiki/Building_Debian_Packages_-_The_Easy_Way
- https://hub.docker.com/r/koha/koha-dpkg
- https://gitlab.com/minusdavid/koha-deb-builder-docker

Signed-off-by: Mason James <mtj@kohaaloha.com>
Comment 8 Mason James 2020-05-18 11:00:14 UTC
(In reply to David Cook from comment #6)
> Upping this one to critical as you can't build packages until it's fixed

i confirm this bug and fix, thanks David
Comment 9 Tomás Cohen Arazi 2020-05-18 22:11:39 UTC
Tested it with the updated koha/koha-dpkg image:

$ docker run --volume=/Users/tcohen/git/koha:/koha --volume=/Users/tcohen/tmp/debs:/debs --privileged --env VERSION=19.12.00.089 koha/koha-dpkg:master

=>

...
dpkg-deb: building package 'koha' in '../koha_19.12.00.089-1_all.deb'.
dpkg-deb: building package 'koha-common' in '../koha-common_19.12.00.089-1_all.deb'.
dpkg-deb: building package 'koha-deps' in '../koha-deps_19.12.00.089-1_all.deb'.
dpkg-deb: building package 'koha-perldeps' in '../koha-perldeps_19.12.00.089-1_all.deb'.
 dpkg-genbuildinfo
 dpkg-genchanges  >../koha_19.12.00.089-1_amd64.changes
dpkg-genchanges: info: including full source code in upload
 dpkg-source --after-build koha-19.12.00.089
dpkg-buildpackage: info: full upload (original source is included)
I: copying local configuration
I: Copying back the cached apt archive contents
I: unmounting dev/ptmx filesystem
I: unmounting dev/pts filesystem
I: unmounting dev/shm filesystem
I: unmounting proc filesystem
I: unmounting sys filesystem
I: cleaning the build env
I: removing directory /var/cache/pbuilder/build/279 and its subdirectories
I: Current time: Mon May 18 22:07:12 UTC 2020
I: pbuilder-time-stamp: 1589839632
git checkout -- debian/changelog

All good!
Comment 10 Tomás Cohen Arazi 2020-05-18 22:13:06 UTC
Created attachment 105051 [details] [review]
Bug 25068: Fix koha-common.install directives

Bug 24735 removed etc/searchengine/queryparser.yml, which removed
the need for koha-common.install to install the following file:
debian/tmp/etc/koha/searchengine

This patch removes that line and causes the Koha packages to build
correctly again.

To test:
0) Apply patch
1) Build a Koha package using one of the following methods:
- https://wiki.koha-community.org/wiki/Building_Debian_Packages_-_The_Easy_Way
- https://hub.docker.com/r/koha/koha-dpkg
- https://gitlab.com/minusdavid/koha-deb-builder-docker

Signed-off-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 11 Tomás Cohen Arazi 2020-05-18 22:56:13 UTC
I'm seeing an upgrade error. Gimme a few
Comment 12 Tomás Cohen Arazi 2020-05-18 22:58:01 UTC
(In reply to Tomás Cohen Arazi from comment #11)
> I'm seeing an upgrade error. Gimme a few

All good, it was just a warning about /etc/koha/searchengine not being empty, but it died, and now I cannot reproduce. My attempt is to have a  19.11 install (vagrant/stretch64) and then install the packages.
Comment 13 David Cook 2020-05-19 02:31:09 UTC
(In reply to Tomás Cohen Arazi from comment #12)
> (In reply to Tomás Cohen Arazi from comment #11)
> > I'm seeing an upgrade error. Gimme a few
> 
> All good, it was just a warning about /etc/koha/searchengine not being
> empty, but it died, and now I cannot reproduce. My attempt is to have a 
> 19.11 install (vagrant/stretch64) and then install the packages.

I recall seeing that warning when upgrading as well, although I didn't have any errors per se. 

That said, I wonder if Bug 24735 needs a follow-up to remove the /etc/koha/searchengine/queryparser.yaml file if it exists... as otherwise that file will just clutter up existing installs.
Comment 14 Tomás Cohen Arazi 2020-05-19 02:58:00 UTC
(In reply to David Cook from comment #13)
> (In reply to Tomás Cohen Arazi from comment #12)
> > (In reply to Tomás Cohen Arazi from comment #11)
> > > I'm seeing an upgrade error. Gimme a few
> > 
> > All good, it was just a warning about /etc/koha/searchengine not being
> > empty, but it died, and now I cannot reproduce. My attempt is to have a 
> > 19.11 install (vagrant/stretch64) and then install the packages.
> 
> I recall seeing that warning when upgrading as well, although I didn't have
> any errors per se. 
> 
> That said, I wonder if Bug 24735 needs a follow-up to remove the
> /etc/koha/searchengine/queryparser.yaml file if it exists... as otherwise
> that file will just clutter up existing installs.

I had a follow-up to submit this one, but then I thought we *could* be wrong on deleting it.
Comment 15 David Cook 2020-05-19 03:04:12 UTC
(In reply to Tomás Cohen Arazi from comment #14)
> I had a follow-up to submit this one, but then I thought we *could* be wrong
> on deleting it.

Yeah I wonder that too. It doesn't feel great deleting config files :/
Comment 16 Martin Renvoize 2020-05-19 07:30:28 UTC
Nice work everyone!

Pushed to master for 20.05
Comment 17 Joy Nelson 2020-05-19 23:16:23 UTC
missing dependencies- not backported to 19.11.x