Bug 33168 - Timeline on "About Koha" is not working for package installs
Summary: Timeline on "About Koha" is not working for package installs
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Packaging (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: David Cook
QA Contact: Marcel de Rooy
URL:
Keywords: rel_21_05_candidate, rel_21_11_candidate
Depends on:
Blocks:
 
Reported: 2023-03-08 12:37 UTC by Laura Escamilla
Modified: 2023-12-28 20:44 UTC (History)
9 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.05.00,22.11.05,22.05.12,21.11.20


Attachments
Bug 33168: Prevent debhelper from renaming history.txt to changelog (1.29 KB, patch)
2023-03-21 23:11 UTC, David Cook
Details | Diff | Splinter Review
Bug 33168: Prevent debhelper from renaming history.txt to changelog (1.33 KB, patch)
2023-03-22 04:03 UTC, Mason James
Details | Diff | Splinter Review
Bug 33168: Prevent debhelper from renaming history.txt to changelog (1.43 KB, patch)
2023-03-24 08:22 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 33168: swap spaces for tabs [follow-up] (857 bytes, patch)
2023-03-31 14:24 UTC, Mason James
Details | Diff | Splinter Review
Bug 33168: (follow-up) Swap spaces for tabs (905 bytes, patch)
2023-03-31 19:27 UTC, David Nind
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Laura Escamilla 2023-03-08 12:37:05 UTC
When viewing the timeline tab on the "About Koha" page, the error: "Could not read the history.txt file. Please make sure <docdir> is correctly defined in koha-conf.xml." shows up.
Comment 1 Joseph Alway 2023-03-08 14:35:04 UTC
The historty.txt file is missing. Not sure why, but it is. In an older version of Koha, the file is there. In the current install it's just missing. Haven't gotten around to checking the Virtual Machine I was testing, before our current install. It may just be an issue with our install?
Comment 2 Katrin Fischer 2023-03-09 21:29:46 UTC
I have checked in master on ktd and there the about page works as expected. Could this be a packaging problem? If he file was not copied into the right place in the file structure, that could explain the issue.

The file is definitely present in the dev environment:
/kohadevbox/koha/docs/history.txt
Comment 3 Laura Escamilla 2023-03-13 16:28:18 UTC
(In reply to Katrin Fischer from comment #2)
> I have checked in master on ktd and there the about page works as expected.
> Could this be a packaging problem? If he file was not copied into the right
> place in the file structure, that could explain the issue.
> 
> The file is definitely present in the dev environment:
> /kohadevbox/koha/docs/history.txt

It definitely works on master in the KTD but all our partners are having this issue. I will look into it further. Thanks!
Comment 4 Mason James 2023-03-14 04:43:01 UTC
(In reply to Laura Escamilla from comment #0)
> When viewing the timeline tab on the "About Koha" page, the error: "Could
> not read the history.txt file. Please make sure <docdir> is correctly
> defined in koha-conf.xml." shows up.

hi Laura
this indicates that you seem to be missing a <docdir> definition

in a koha installed with a debian package, the following line should exist in your /etc/koha/sites/mykoha1/koha-conf.xml

  <docdir>/usr/share/doc/koha-common</docdir>


fyi: the history file should exist at /usr/share/doc/koha-common/history.txt
Comment 5 Katrin Fischer 2023-03-15 15:15:05 UTC
Hi Mason, 

it looks like this is not a singular/one installation issue. Our package installations of 22.11 are missing the file as well. It looks like a general issue with the package installations.
Comment 6 Mason James 2023-03-16 02:30:37 UTC
(In reply to Katrin Fischer from comment #5)
> Hi Mason, 
> 
> it looks like this is not a singular/one installation issue. Our package
> installations of 22.11 are missing the file as well. It looks like a general
> issue with the package installations.

hmm, the file exists in 22.11.03 package

----------

# apt install  koha-common

# apt policy koha-common
koha-common:
  Installed: 22.11.03-1
  Candidate: 22.11.03-1
  Version table:
 *** 22.11.03-1 500
        500 http://debian.koha-community.org/koha 22.11/main amd64 Packages
        100 /var/lib/dpkg/status

# dpkg -L   koha-common | grep history.txt
/usr/share/doc/koha-common/history.txt

# ls -l /usr/share/doc/koha-common/history.txt
-rw-r--r-- 1 root root 113207 Mar  1 01:54 /usr/share/doc/koha-common/history.txt
Comment 7 David Cook 2023-03-16 03:57:02 UTC
(In reply to Laura Escamilla from comment #0)
> When viewing the timeline tab on the "About Koha" page, the error: "Could
> not read the history.txt file. Please make sure <docdir> is correctly
> defined in koha-conf.xml." shows up.

Can we get information on what version of Koha this is happening?
Comment 8 Katrin Fischer 2023-03-16 15:24:51 UTC
Happens on our freshly packaged 22.11.03
Comment 9 David Cook 2023-03-17 02:47:47 UTC
(In reply to Katrin Fischer from comment #8)
> Happens on our freshly packaged 22.11.03

I'm going to set the version to 22.11 as 22.05 is fine and master ktd is OK.
Comment 10 David Cook 2023-03-17 02:53:44 UTC
I'm a bit behind these days but I should be looking at 22.11 Deb package installs in the near future...
Comment 11 Katrin Fischer 2023-03-17 07:55:11 UTC
We have <docdir>/usr/share/doc/koha-common</docdir> and history.txt doesn't exist.
Comment 12 Katrin Fischer 2023-03-17 08:23:37 UTC
(In reply to David Cook from comment #9)
> (In reply to Katrin Fischer from comment #8)
> > Happens on our freshly packaged 22.11.03
> 
> I'm going to set the version to 22.11 as 22.05 is fine and master ktd is OK.

I believe it's a packages thing because the file is not ending up where it should be, so it's not being copied/installed into the right spot.
Comment 13 Mason James 2023-03-19 05:26:36 UTC
(In reply to Katrin Fischer from comment #12)
> (In reply to David Cook from comment #9)
> > (In reply to Katrin Fischer from comment #8)
> > > Happens on our freshly packaged 22.11.03
> > 
> > I'm going to set the version to 22.11 as 22.05 is fine and master ktd is OK.
> 
> I believe it's a packages thing because the file is not ending up where it
> should be, so it's not being copied/installed into the right spot.

whats the output of?

# dpkg -L   koha-common | grep history.txt
Comment 14 David Cook 2023-03-19 22:26:38 UTC
Katrin: Is this a vanilla Debian package or is this one that you packaged?

I'm running a vanilla 22.05 and I see the history.txt, but I've looked again at my custom 21.11 and I do *not* see the history.txt.

Now I would've sworn that my 21.11 instances had a history.txt before, but looking today it is certainly gone. (However, I still see teams.yaml and contributors.yaml in /usr/share/doc/koha-common...)
Comment 15 David Cook 2023-03-19 22:33:03 UTC
# dpkg -l koha-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-===============================================
ii  koha-common    22.05.08-2   all          integrated (physical) library management system

# dpkg -L   koha-common | grep "history.txt"
/usr/share/doc/koha-common/history.txt

-----------------------------------------------------

# dpkg -l koha-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version                                  Architecture Description
+++-==============-========================================-============-===============================================
ii  koha-common    22.12.00-9~git+20230228053212.66e9bc71-1 all          integrated (physical) library management system

# dpkg -L   koha-common | grep "history.t

--------------------------------------------------
Comment 16 David Cook 2023-03-19 22:34:10 UTC
Checking more custom versions...

I have it in custom 20.11 but I do not have it in custom 21.11...
Comment 17 David Cook 2023-03-19 22:51:08 UTC
Interestingly my custom 21.11 had /usr/share/doc/koha-common/history.txt back in March 2022 but not in March 2023...

Not noticing anything obvious in the git repo that would've caused a change...

I think I may have made some build machine changes during that time so maybe that's related...
Comment 18 Katrin Fischer 2023-03-19 22:57:02 UTC
We built our own packages, although we are still very close to standard Koha. But Laura initially reported this, so it might be interesting to check with her too.
Comment 19 David Cook 2023-03-19 23:30:46 UTC
If I make a tarball from custom 21.11, I see docs/history.txt is in there...

But when I build from that same branch, the history.txt file is not in the .deb package. I can confirm that using 'dpkg -c' on the .deb file. 

--

It looks like it's being installed by dh_auto_install into the '/debian/tmp' build directory... 

Installing /home/koha/build/koha-21.11.XX-XXXX+20230320095638.5d5cedcd/debian/tmp/usr/share/koha/doc/teams.yaml
Installing /home/koha/build/koha-21.11.XX-XXXX+20230320095638.5d5cedcd/debian/tmp/usr/share/koha/doc/contributors.yaml
Installing /home/koha/build/koha-21.11.XX-XXXX+20230320095638.5d5cedcd/debian/tmp/usr/share/koha/doc/history.txt

However in debian/rules, we nuke /usr/share/koha/doc since we don't need it in /usr/share/koha... so that's irrelevant. Just shows that it does exist at that point in time...

--

I don't know. It's so weird that teams.yaml and contributors.yaml get included but history.txt does not.
Comment 20 Mason James 2023-03-21 12:21:22 UTC
(In reply to David Cook from comment #15)
> ii  koha-common    22.12.00-9~git+20230228053212.66e9bc71-1 all         
> integrated (physical) library management system
> 
> # dpkg -L   koha-common | grep "history.t
> 
> --------------------------------------------------

seems to be working ok on the latest master/dev branch

root@deb12:~# rm /usr/share/doc/koha-common/history.txt

root@deb12:~# apt install koha-common 
Unpacking koha-common (22.12.00-14~git+20230321055412.0fb86ccd-1) over (22.12.00-13~git+20230318103210.afdc1690-1) ...

root@deb12:~# ls -l  /usr/share/doc/koha-common/history.txt
-rw-r--r-- 1 root root 113207 Mar 21 06:00 /usr/share/doc/koha-common/history.txt
Comment 21 Mason James 2023-03-21 12:55:15 UTC
(In reply to Mason James from comment #20)
> (In reply to David Cook from comment #15)
> > ii  koha-common    22.12.00-9~git+20230228053212.66e9bc71-1 all         
> > integrated (physical) library management system
> > 
> > # dpkg -L   koha-common | grep "history.t
> > 
> > --------------------------------------------------
> 
> seems to be working ok on the latest master/dev branch

hmm, i think ive traced the problem to using deb9 vs deb10 pbuilder versions, (wtf)
Comment 22 David Cook 2023-03-21 22:23:30 UTC
(In reply to Mason James from comment #21)
> hmm, i think ive traced the problem to using deb9 vs deb10 pbuilder
> versions, (wtf)

Yeah, I think the problem surfaced when I switched from building on deb9 Stretch to deb11 Bullseye. 

I'm not using pbuilder so I imagine it's a lower level component causing the problem.

I'm wondering if "history.txt" is a magic word to newer versions of debuild (or another component), and if the solution is to just rename the file. 

Apparently dh_installchangelogs looks for history.txt...

Ahhhhhh guess what file is installed as /usr/share/doc/koha-common/changelog.gz? Our old friend history.txt!

"In non-native packages, dh_installchangelogs will first look for changelog files installed by the upstream build system into usr/share/doc/package (of the package build directory) and rename the most likely candidate (if any) to usr/share/doc/package/changelog."

https://manpages.debian.org/testing/debhelper/dh_installchangelogs.1.en.html
Comment 23 David Cook 2023-03-21 22:25:13 UTC
Looking back at 20.11, it looks like /usr/share/doc/koha-common/changelog.gz and /usr/share/doc/koha-common/history.txt are the same (except changelog.gz is obviously compressed)
Comment 24 David Cook 2023-03-21 22:29:08 UTC
(In reply to David Cook from comment #23)
> Looking back at 20.11, it looks like /usr/share/doc/koha-common/changelog.gz
> and /usr/share/doc/koha-common/history.txt are the same (except changelog.gz
> is obviously compressed)

The stretch version of dh_installchangelogs doesn't say anything about renaming: 
https://manpages.debian.org/stretch/debhelper/dh_installchangelogs.1.en.html

But the docs change for deb10 Buster: https://manpages.debian.org/buster/debhelper/dh_installchangelogs.1.en.html

Of course, I might be wrong. The documentation may have been updated while the behaviour stayed the same. Even in "stretch", there is a "-k" option for keeping the original name of the upstream changelog...
Comment 25 David Cook 2023-03-21 22:38:45 UTC
Ok if we compare different versions of dh_installchangelogs (thank you open source), we can see that the rename didn't happen back in the Stretch version:

https://github.com/Debian/debhelper/blob/debian/10.2.5/dh_installchangelogs

https://github.com/Debian/debhelper/blob/master/dh_installchangelogs

Looks like the change was introduced in debian 11.3 I think:
https://github.com/Debian/debhelper/commit/406bd37eb4692e172fbf6c3c390cb4f50d8eaae9

What a nice little breaking change hehe
Comment 26 David Cook 2023-03-21 22:44:00 UTC
So I reckon changing from history.txt to something else and then updating Koha to look for a different filename is the way to go.

However, we'd lose the current incarnation of changelog.gz in the process. 

That said, typically changelog.gz  contains the contents of the Debian package changelog anyway, and we already have that via changelog.Debian.gz, so I think it's all good.
Comment 27 David Cook 2023-03-21 22:46:50 UTC
Hmm I bet a number of the release tools look for history.txt. Maybe renaming is actually going to be more work...
Comment 28 David Cook 2023-03-21 22:52:30 UTC
I'm going to try out some packaging options...

First up:

override_dh_installchangelogs:
    dh_installchangelogs --exclude=history.txt
Comment 29 David Cook 2023-03-21 23:07:26 UTC
As predicted, we regain /usr/share/doc/koha-common/history.txt at the expense of /usr/share/doc/koha-common/changelog.gz

I think that's fine. 

I've noticed a number of packages that have "changelog.Debian.gz" but not "changelog.gz" or their changelog.gz is the same as "changelog.Debian.gz".
Comment 30 David Cook 2023-03-21 23:11:37 UTC
Created attachment 148497 [details] [review]
Bug 33168: Prevent debhelper from renaming history.txt to changelog

This change prevents debhelper from renaming history.txt to changelog,
since that renaming was breaking the Timeline feature in Koha.

Another option would've been to rename history.txt to something else
but that filename is already embedded in Koha and the Koha release
tools, so this seems the safer option.

Test plan:
0. Apply patch
1. Build Debian package
2. Confirm that /usr/share/doc/koha-common/history.txt is created
and /usr/share/doc/koha-common/changelog.gz is not created
Comment 31 David Cook 2023-03-21 23:23:53 UTC
In terms of "Version" affected, I suppose there is no harm in backporting this one...
Comment 32 Mason James 2023-03-22 04:03:30 UTC
Created attachment 148501 [details] [review]
Bug 33168: Prevent debhelper from renaming history.txt to changelog

This change prevents debhelper from renaming history.txt to changelog,
since that renaming was breaking the Timeline feature in Koha.

Another option would've been to rename history.txt to something else
but that filename is already embedded in Koha and the Koha release
tools, so this seems the safer option.

Test plan:
0. Apply patch
1. Build Debian package
2. Confirm that /usr/share/doc/koha-common/history.txt is created
and /usr/share/doc/koha-common/changelog.gz is not created

Signed-off-by: Mason James <mtj@kohaaloha.com>
Comment 33 Mason James 2023-03-22 04:04:12 UTC
> Signed-off-by: Mason James <mtj@kohaaloha.com>

tested as working on deb10
Comment 34 Katrin Fischer 2023-03-22 12:12:33 UTC
It looks like this required quite a bit of detectice work - thanks a lot!
Comment 35 David Cook 2023-03-22 22:22:07 UTC
(In reply to Katrin Fischer from comment #34)
> It looks like this required quite a bit of detectice work - thanks a lot!

A hunch and some lucky searching heh

Since Mason and I have already worked on this one, how do we move this along through QA?
Comment 36 Jonathan Druart 2023-03-23 10:52:07 UTC
1. Build Debian package

What's the easiest for that steps (in March 2023)?
Comment 37 Mason James 2023-03-23 12:30:34 UTC
(In reply to Jonathan Druart from comment #36)
> 1. Build Debian package
> 
> What's the easiest for that steps (in March 2023)?

hi Jonathan

try this wiki page, its been updated recently
 https://wiki.koha-community.org/wiki/Building_Debian_Packages_-_The_Easy_Way


update us, if you have any problems :)
Comment 38 Marcel de Rooy 2023-03-24 08:22:08 UTC
Created attachment 148632 [details] [review]
Bug 33168: Prevent debhelper from renaming history.txt to changelog

This change prevents debhelper from renaming history.txt to changelog,
since that renaming was breaking the Timeline feature in Koha.

Another option would've been to rename history.txt to something else
but that filename is already embedded in Koha and the Koha release
tools, so this seems the safer option.

Test plan:
0. Apply patch
1. Build Debian package
2. Confirm that /usr/share/doc/koha-common/history.txt is created
and /usr/share/doc/koha-common/changelog.gz is not created

Signed-off-by: Mason James <mtj@kohaaloha.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 39 Marcel de Rooy 2023-03-24 08:23:42 UTC
In full trust :)

And covered by what I found here:

The upstream changelog is automatically found by the dh_installchangelogs using the case insensitive match of its file name to changelog, changes, changelog.txt, changes.txt, history, history.txt, or changelog.md and searched in the ./ doc/ or docs/ directories.
Comment 40 Tomás Cohen Arazi 2023-03-31 10:17:56 UTC
Pushed to master for 23.05.

Nice work everyone, thanks!
Comment 41 Mason James 2023-03-31 14:24:52 UTC
Created attachment 149038 [details] [review]
Bug 33168: swap spaces for tabs [follow-up]
Comment 42 David Nind 2023-03-31 19:27:55 UTC
Created attachment 149055 [details] [review]
Bug 33168: (follow-up) Swap spaces for tabs

Signed-off-by: David Nind <david@davidnind.com>
Comment 43 David Nind 2023-03-31 19:33:33 UTC
I tested the follow-up patch as follows using KTD:

1. Open debian/rules in a text editor with tabs and spaces displayed.
2. Note that on lines 51 and 52 that spaces are used instead of tabs. 
3. Apply the follow-up patch only (the main patch is already in master).
4. Note that the spaces in step 2 are now tabs.

I hope this is sufficient.

I also amended the commit message to follow the title conventions for follow-up patches.

Note: I also noticed that for lines 43 and 44 that spaces are used after the first tab - do these need to be fixed as well?
Comment 44 Mason James 2023-04-01 08:46:21 UTC
(In reply to David Nind from comment #43)
> Note: I also noticed that for lines 43 and 44 that spaces are used after the
> first tab - do these need to be fixed as well?

hi David
thanks for the sign-off

lines 43 and 44 are OK :)
Comment 45 Jonathan Druart 2023-04-04 09:27:06 UTC
(In reply to David Nind from comment #42)
> Created attachment 149055 [details] [review] [review]
> Bug 33168: (follow-up) Swap spaces for tabs
> 
> Signed-off-by: David Nind <david@davidnind.com>

Pushed to master for 23.05.00.
Comment 46 Matt Blenkinsop 2023-04-11 11:25:07 UTC
Nice work everyone!

Pushed to stable for 22.11.x
Comment 47 Lucas Gass 2023-04-24 17:24:41 UTC
Backported to 22.05.x for upcoming 22.05.12
Comment 48 Mason James 2023-05-03 02:00:10 UTC
(In reply to Lucas Gass from comment #47)
> Backported to 22.05.x for upcoming 22.05.12

can we get this backported to 21.11 and 21.05 please
Comment 49 Arthur Suzuki 2023-05-03 19:05:59 UTC
applied to 21.11.x for 21.11.20