Bug 21028 - www/history.t is failing on jenkins for Ubuntu 18.04
Summary: www/history.t is failing on jenkins for Ubuntu 18.04
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low major (vote)
Assignee: Chris Cormack
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks: 23850
  Show dependency treegraph
 
Reported: 2018-07-02 15:17 UTC by Jonathan Druart
Modified: 2020-06-04 20:32 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2018-07-02 15:17:15 UTC
t/db_dependent/www/history.t is failing with the following error on Ubuntu 18.04 (at least with the image from koha-testing-docker):

koha_1       | #   Failed test 'Cyrillic characters with umlauts show correctly on the history page.'
koha_1       | #   at t/db_dependent/www/history.t line 68.

It is *not* an encoding issue, the problem is that history.txt is not found:

Could not read the history.txt file. Please make sure <docdir> is correctly defined in koha-conf.xml.

From koha-conf.xml:
  <docdir>/usr/share/doc/koha-common</docdir>

Indeed this location does not contain history.txt:
root@95f30c756223:/kohadevbox/koha# ls /usr/share/doc/koha-common
changelog.Debian.gz  copyright


I have no idea if it is a koha issue, a packaging issue, or a koha-testing issue.

Does anyone has an idea?

https://jenkins.koha-community.org/job/Koha_18.05_U18/14/consoleFull
Comment 1 Martin Renvoize 2018-07-02 16:19:59 UTC
Glad you created a bug for this, I was also coming up empty and about to do the same when I saw yours come in :).

Afraid I can't add much more than that.
Comment 2 Jonathan Druart 2018-07-13 11:15:15 UTC
Upping severity
Comment 3 Mirko Tietgen 2018-07-16 10:57:31 UTC
The Bionic nightly package does contain /usr/share/doc/koha-common/history.txt. Log mentions [https://apt.abunchofthings.net/koha-nightly/unstable-bionic-log.txt]

> cp docs/history.txt blib/DOC_DIR/history.txt

> Installing /build/koha-18.06.01~git+20180716093736.12fd02af/debian/tmp/usr/share/koha/doc/history.txt

> # exclude history.txt from compression as about.pl needs it uncompressed
> dh_compress --exclude=history.txt


If the test uses /usr/share/doc/koha-common/history.txt, I suppose it is necessary to copy that file from /kohadevbox/koha/doc/, like the Jenkins log shows for Debian scripts? Or rather use /kohadevbox/koha/doc/history.txt instead.

But if that is the problem, I would expect the test to fail for other distros.
Comment 4 Jonathan Druart 2018-07-16 14:58:28 UTC
It's a www test, so it means the interface is hit. It's not test related.

So I am expecting about.pl to not display the history.txt content (tab Timeline) on bionic. Could someone confirm?

There is something hidden we should understand asap, it could have other problematic side-effects. Or Maybe none... :)
Comment 5 Jonathan Druart 2018-07-16 16:31:30 UTC
Tested on bionic:

% sudo koha-create xxx
% sudo koha-shell xxx
% grep docdir $KOHA_CONF 
 <docdir>/usr/share/doc/koha-common</docdir>

% ls /usr/share/doc/koha-common|grep history.txt
history.txt
Comment 6 Tomás Cohen Arazi 2018-07-16 19:59:31 UTC
(In reply to Jonathan Druart from comment #5)
> Tested on bionic:
> 
> % sudo koha-create xxx
> % sudo koha-shell xxx
> % grep docdir $KOHA_CONF 
>  <docdir>/usr/share/doc/koha-common</docdir>
> 
> % ls /usr/share/doc/koha-common|grep history.txt
> history.txt

I will try testing this in the docker environment, which seems to be to blame right now.
Comment 7 Jonathan Druart 2018-08-03 12:45:07 UTC
Tested the docker image, apt-get remote koha --purge, then:


root@a33753dce67d:/kohadevbox# apt-get install koha-common
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  mysql-server | virtual-mysql-server letsencrypt
The following NEW packages will be installed:
  koha-common
0 upgraded, 1 newly installed, 0 to remove and 37 not upgraded.
Need to get 31.5 MB of archives.
After this operation, 470 MB of additional disk space will be used.
Get:1 http://apt.abunchofthings.net/koha-nightly unstable/main amd64 koha-common all 18.06.01~git+20180803002557.c6b50f0b-1 [31.5 MB]
Fetched 31.5 MB in 13s (2,385 kB/s)                                                                                                                                                                               
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package koha-common.
(Reading database ... 41209 files and directories currently installed.)
Preparing to unpack .../koha-common_18.06.01~git+20180803002557.c6b50f0b-1_all.deb ...
Unpacking koha-common (18.06.01~git+20180803002557.c6b50f0b-1) ...
Setting up koha-common (18.06.01~git+20180803002557.c6b50f0b-1) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Upgrading database schema for kohadev
root@a33753dce67d:/kohadevbox#
root@a33753dce67d:/kohadevbox# updatedb
root@a33753dce67d:/kohadevbox# locate history.txt
/kohadevbox/koha/docs/history.txt
Comment 8 Tomás Cohen Arazi 2018-08-03 12:50:06 UTC
(In reply to Jonathan Druart from comment #7)
> Tested the docker image, apt-get remote koha --purge, then:
> 
> 
> root@a33753dce67d:/kohadevbox# apt-get install koha-common
> Reading package lists... Done
> Building dependency tree       
> Reading state information... Done
> Suggested packages:
>   mysql-server | virtual-mysql-server letsencrypt
> The following NEW packages will be installed:
>   koha-common
> 0 upgraded, 1 newly installed, 0 to remove and 37 not upgraded.
> Need to get 31.5 MB of archives.
> After this operation, 470 MB of additional disk space will be used.
> Get:1 http://apt.abunchofthings.net/koha-nightly unstable/main amd64
> koha-common all 18.06.01~git+20180803002557.c6b50f0b-1 [31.5 MB]
> Fetched 31.5 MB in 13s (2,385 kB/s)                                         
> 
> debconf: delaying package configuration, since apt-utils is not installed
> Selecting previously unselected package koha-common.
> (Reading database ... 41209 files and directories currently installed.)
> Preparing to unpack
> .../koha-common_18.06.01~git+20180803002557.c6b50f0b-1_all.deb ...
> Unpacking koha-common (18.06.01~git+20180803002557.c6b50f0b-1) ...
> Setting up koha-common (18.06.01~git+20180803002557.c6b50f0b-1) ...
> invoke-rc.d: could not determine current runlevel
> invoke-rc.d: policy-rc.d denied execution of start.
> Upgrading database schema for kohadev
> root@a33753dce67d:/kohadevbox#
> root@a33753dce67d:/kohadevbox# updatedb
> root@a33753dce67d:/kohadevbox# locate history.txt
> /kohadevbox/koha/docs/history.txt

Yes somehow installing in the docker image we are using doesn't put the file where we expect it. This is not reproducible in a regular VM (vagrant init ubuntu/bionic64; vagrant up).

I vote for actually setting <docdir> to the git repo's history file.
Comment 9 Jonathan Druart 2018-08-03 13:47:54 UTC
root@a33753dce67d:/kohadevbox# for l in `head -n 100 DEBIAN/md5sums|tr -s ' ' | cut -d' ' -f2`; do ls "/$l" > /dev/null 2>&1; if [[ $? != 0 ]]; then echo $l; fi; done
usr/share/doc/koha-common/LEEME.Debian
usr/share/doc/koha-common/README.Debian
usr/share/doc/koha-common/changelog.gz
usr/share/doc/koha-common/history.txt
usr/share/doc/koha-common/release_notes_30000.txt.gz
usr/share/doc/koha-common/release_notes_30000alpha.txt.gz
usr/share/doc/koha-common/release_notes_30000beta.txt.gz
usr/share/doc/koha-common/release_notes_30000stableRC1.txt.gz
usr/share/doc/koha-common/release_notes_30001.txt.gz
usr/share/doc/koha-common/release_notes_30002.txt.gz
usr/share/doc/koha-common/release_notes_30003.txt.gz
usr/share/doc/koha-common/release_notes_30004.txt.gz
usr/share/doc/koha-common/release_notes_30005.txt.gz
usr/share/doc/koha-common/release_notes_3_10_0.txt.gz
usr/share/doc/koha-common/release_notes_3_12_0.txt.gz
usr/share/doc/koha-common/release_notes_3_14_0.txt.gz
usr/share/doc/koha-common/release_notes_3_16_0.txt.gz
usr/share/doc/koha-common/release_notes_3_18_0.txt.gz
usr/share/doc/koha-common/release_notes_3_20_0.txt.gz
usr/share/doc/koha-common/release_notes_3_22_0.txt.gz
usr/share/doc/koha-common/release_notes_3_2_0.txt.gz
usr/share/doc/koha-common/release_notes_3_4_0.txt.gz
usr/share/doc/koha-common/release_notes_3_6_0.txt.gz
usr/share/doc/koha-common/release_notes_3_8_0.txt.gz

So yes, it's only the docdir, that's very weird.
Comment 10 Jonathan Druart 2018-08-06 14:39:07 UTC
We did not find the real cause of this issue, but it has been "fixed" (workarounded) at koha-testing-docker level, see https://gitlab.com/koha-community/koha-testing-docker/merge_requests/77
Comment 11 Martin Renvoize 2019-06-07 08:22:35 UTC
It feels like it could be some fun with dh_compress.. history.txt (until very recently) was the only file mentioned in the exclude list there.. I can't find any other reason it may be treated differently to other files.

I want to check what the two new files added to the exclude list look like on Ubuntu to test my theory.

Reopening as the fix in koha-testing-docker is no longer working since we tidied up that project.. time to revisit the core issue here I tihnk.
Comment 12 Jonathan Druart 2019-06-08 17:46:28 UTC
(In reply to Martin Renvoize from comment #11)
> I can't find any other reason it may be treated differently to other files.

From commit b7cd95bcc909f7db330b5acbfcf9e8f817328a67
"""
    Bug 10802: make Debian package install history.txt

    While bug 8911 fixed a problem regarding history.txt, a proper fix for
    packages was missing.
    This patch does three things:

     - Adds docs/history.txt to debian/koha-common.docs so it gets installed.
     - Moves the line installing the release notes from debian/koha-common.install
       to debian/koha-common.docs.
     - Adds an override_dh_compress entry in debian/rules that excludes
       history.txt from the dh_compress routine [1]
[...]

    [1] As 'man dh_compress' says, it is needed as every file considered for
        the doc dir that is plain text and is more than 4k size will be
        gzipped unless stated otherwise. We need it uncompressed for about.pl.
"""
Comment 13 Martin Renvoize 2019-06-13 16:26:31 UTC
Still baffled.. but I've re-implemented the hack we did last time but this time added it as a step in the Ubuntu Dockerfile.. Closing again until it raises it's ugly head once more.