Hi folks, As per this Mattermost conversation: https://chat.koha-community.org/koha-community/pl/ft6e47akutn6fbdg5tpwab5zbw I've identified an issue where running t/db_dependent/Search.t fails on AARCH64 machines. Upon investigation, I've found that in the default zebra-biblios-dom.cfg and zebra-authorities-dom.cfg files, the module search path does not include as many architectures as it should. I have rectified this, by copying the lines found in the debian/templates folder to the relevant config files mentioned in the last paragraph. Could I ask someone to check the patch I'm about to push, so we can get a quick conclusion to this bug? Ta, Jake
Created attachment 183783 [details] [review] Bug 40304: Zebrasrv configs now work on ARM CPUs This patch adds more module search paths to the zebra-biblios-dom.cfg and zebra-authorities-dom.cfg, so that a larger range of CPU architectures will work with Zebra search unit tests. This will not change Debian package installs, but it will fix any git-based installs. TO TEST (you'll need access to an ARM-based CPU, and, preferably, an x86-64 machine): ON THE ARM-BASED MACHINE: a) start ktd, then ktd --shell b) in ktd, run: prove -v t/db_dependent/Search.t c) notice how the test now fails APPLY PATCH d) rerun step b e) notice how the test now passes ON THE X86-64-BASED MACHINE: f) run steps a-e g) notice how the test passes, both before and after applying the patch SIGN OFF
Created attachment 183784 [details] [review] Bug 40304: Zebrasrv configs now work on ARM CPUs This patch adds more module search paths to the zebra-biblios-dom.cfg and zebra-authorities-dom.cfg, so that a larger range of CPU architectures will work with Zebra search unit tests. This will not change Debian package installs, but it will fix any git-based installs. TO TEST (you'll need access to an ARM-based CPU, and, preferably, an x86-64 machine): ON THE ARM-BASED MACHINE: a) start ktd, then ktd --shell b) in ktd, run: prove -v t/db_dependent/Search.t c) notice how the test now fails APPLY PATCH d) rerun step b e) notice how the test now passes ON THE X86-64-BASED MACHINE: f) run steps a-e g) notice how the test passes, both before and after applying the patch SIGN OFF Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Sorry to do this guys but I think that this introduces a regression. The path /usr/lib64/idzebra-2.0/modules is removed by the patch, but that's the path used by OSes like OpenSUSE (and I think Fedora/RHEL as well although I don't have any of those on-hand atm).
Hi David, Thanks for looking. If this is indeed a problem, Debian packages will also have this broken behaviour. Perhaps this isn't an issue, as Debian packages only go on Debian systems, but nonetheless, perhaps it wouldn't hurt fixing this in both places? I shall test both eventualities, and push a new patch shortly. Ta, Jake.
Created attachment 183831 [details] [review] Bug 40304: Zebrasrv configs now work on ARM CPUs This patch adds more module search paths to the zebra-biblios-dom.cfg and zebra-authorities-dom.cfg, so that a larger range of CPU architectures will work with Zebra search unit tests. This will not change Debian package installs, but it will fix any git-based installs. TO TEST (you'll need access to an ARM-based CPU, and, preferably, an x86-64 machine): ON THE ARM-BASED MACHINE: a) start ktd, then ktd --shell b) in ktd, run: prove -v t/db_dependent/Search.t c) notice how the test now fails APPLY PATCH d) rerun step b e) notice how the test now passes ON THE X86-64-BASED MACHINE: f) run steps a-e g) notice how the test passes, both before and after applying the patch SIGN OFF Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
That's that change implemented. I've put /usr/lib64/idzebra-2.0/modules first in the sequence, as most folk will be running x86-64, and so it makes sense for Zebrasrv to search there first. Tested on ARM64 and x86-64. If anyone has a MIPS or POWERPC platform lying around, it would be great to get a thumbs up from you folk too – though, it isn't essential at this stage. Ta, Jake.
(In reply to Jake Deery from comment #4) > Thanks for looking. If this is indeed a problem, Debian packages will also > have this broken behaviour. Perhaps this isn't an issue, as Debian packages > only go on Debian systems, but nonetheless, perhaps it wouldn't hurt fixing > this in both places? (In reply to Jake Deery from comment #6) > That's that change implemented. I've put /usr/lib64/idzebra-2.0/modules > first in the sequence, as most folk will be running x86-64, and so it makes > sense for Zebrasrv to search there first. > > Tested on ARM64 and x86-64. If anyone has a MIPS or POWERPC platform lying > around, it would be great to get a thumbs up from you folk too – though, it > isn't essential at this stage. Sorry I should've been more precise before. On 64bit Debian/Ubuntu, you'll find the Zebra modules in /usr/lib/x86_64-linux-gnu/idzebra-2.0/modules/. On 64bit OpenSUSE/SLES (and maybe the Red Hat systems), you'll find them in /usr/lib64/idzebra-2.0/modules/. So the Debian packages didn't have broken behaviour, since - as you say - they'd only be on Debian-based systems. So it would make sense for them to have Debian-specific paths. Your patch that I failed removed /usr/lib64 from the generic Zebra conf, which would create a regression on non-Debian-based systems. I won't fail this latest patch, but you might want to consider not including /usr/lib64 in the Debian package conf (I like the idea of using /usr/lib/x86_64-linux-gnu/idzebra-2.0/modules first in the modulePath config though - as you say it makes sense to search there first.). But the ones in the "./etc" path should support everything since they could be deployed anywhere.
If we want to double-check RPM packaged Zebra, we can look at the following: https://github.com/indexdata/idzebra/blob/master/idzebra.spec There we see the use of the %_libdir macro, so then we do some Googling. I won't include the links here because then Bugzilla will auto-ban me for being a spammer :). But for opensuse, it shows %_libdir goes into /usr/lib(64) For Fedora, it says %_libdir goes to /usr/%{_lib} and %{_lib} is lib64 (or just lib on 32bit platforms. Of course, they're probably assuming x86 for all those... hmmm. -- How'd you install the Zebra on the ARM system?
Okie dokes... I looked up libidzebra-2.0-mod-dom on packages.debian.org and looking at the "list of files" I see all the different paths. That's cool. -- Looking again at OpenSUSE/Fedora... I don't think they include the CPU architecture name in the path. They just do 32bit vs 64bit.
Thanks for looking again, David. I did consider leaving the Debian-specific zebra-*-dom.cfg files as they were, but decided to include /usr/lib64, as: * /usr/lib64 does exist on Debian-based systems (even if it isn't used by default) and so may be used if someone decided to compile and install Zebrasrv themselves * It is, from my reading and understanding, a Linux standard path, and so I felt like it should be included for compatibility. Thanks for your time on this. I'll try to get a Mac user's QA on this one today. Ta, Jake.
(In reply to Jake Deery from comment #10) > Thanks for looking again, David. I did consider leaving the Debian-specific > zebra-*-dom.cfg files as they were, but decided to include /usr/lib64, as: > > * /usr/lib64 does exist on Debian-based systems (even if it isn't used by > default) and so may be used if someone decided to compile and install > Zebrasrv themselves > > * It is, from my reading and understanding, a Linux standard path, and so > I felt like it should be included for compatibility. Oh snap, you're right. I think I'd just looked at Debian 11 and Ubuntu 22.04. I looked at Ubuntu 24.04 just now and I see that there is a /usr/lib64. It's not really used for much except it has a symlink from /usr/lib64/ld-linux-x86-64.so.2 to /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2. But still... fair enough! > Thanks for your time on this. I'll try to get a Mac user's QA on this one > today. Oh you know me... I like the weird stuff haha. (I'm also low key interested in Koha having ARM support.) I didn't think that Mac could run Koha though? I thought Mac users run Koha in Linux virtual machines?
Hi David, That's right, but that Linux image is ARM-based Debian 12 (up until about a year ago we were emulating x86-64 due to constraints with Docker – absolute madness). It's been us folk on those ARM-based Macs that noticed this bug in the first place, I think. Sussing out this problem is going to be a huge bonus for running Koha against ARM in Jenkins, too! Ta, Jake.
(In reply to Jake Deery from comment #12) > That's right, but that Linux image is ARM-based Debian 12 (up until about a > year ago we were emulating x86-64 due to constraints with Docker – absolute > madness). > > It's been us folk on those ARM-based Macs that noticed this bug in the first > place, I think. Sussing out this problem is going to be a huge bonus for > running Koha against ARM in Jenkins, too! Yes, of course. I'm not a Mac user so I'd totally forgotten there were ARM-based Macs. Makes sense. From time to time, I get the question about whether or not Koha can run on ARM, and my answer is usually something like "I don't think it can 100%". But I'd love for that to change to "Yes, definitely."
(In reply to David Cook from comment #13) > (In reply to Jake Deery from comment #12) > > That's right, but that Linux image is ARM-based Debian 12 (up until about a > > year ago we were emulating x86-64 due to constraints with Docker – absolute > > madness). > > > > It's been us folk on those ARM-based Macs that noticed this bug in the first > > place, I think. Sussing out this problem is going to be a huge bonus for > > running Koha against ARM in Jenkins, too! > > Yes, of course. I'm not a Mac user so I'd totally forgotten there were > ARM-based Macs. Makes sense. > > From time to time, I get the question about whether or not Koha can run on > ARM, and my answer is usually something like "I don't think it can 100%". > But I'd love for that to change to "Yes, definitely." It does run. Been using an ARM mac daily since 2023 :-D
Created attachment 184240 [details] [review] Bug 40304: Zebrasrv configs now work on ARM CPUs This patch adds more module search paths to the zebra-biblios-dom.cfg and zebra-authorities-dom.cfg, so that a larger range of CPU architectures will work with Zebra search unit tests. This will not change Debian package installs, but it will fix any git-based installs. TO TEST (you'll need access to an ARM-based CPU, and, preferably, an x86-64 machine): ON THE ARM-BASED MACHINE: a) start ktd, then ktd --shell b) in ktd, run: prove -v t/db_dependent/Search.t c) notice how the test now fails APPLY PATCH d) rerun step b e) notice how the test now passes ON THE X86-64-BASED MACHINE: f) run steps a-e g) notice how the test passes, both before and after applying the patch SIGN OFF Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Nice work everyone! Pushed to main for 25.11
Nice work everyone! Pushed to 25.05.x
Pushed to 24.11.x for 24.11.08