Bug 38912

Summary: Elasticsearch record show is a 404 from staff catalog details
Product: Koha Reporter: Eric Phetteplace <ephetteplace>
Component: Searching - ElasticsearchAssignee: Bugs List <koha-bugs>
Status: Failed QA --- QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: david, dcook, fridolin.somers, magnus, mtj, nick, schodkowy.omegi-0r
Version: 24.05Keywords: dependency
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37494
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35618
GIT URL: Change sponsored?: ---
Patch complexity: Trivial patch Documentation contact:
Documentation submission: Text to go in the release notes:
Version(s) released in:
Circulation function:
Attachments: screenshot of 404
Bug 38912: Add option to include '_doc' type in ES record view

Description Eric Phetteplace 2025-01-16 17:17:57 UTC
Created attachment 176694 [details]
screenshot of 404

To recreate:
- Use Elasticsearch as search engine (SearchEngine = Elasticsearch)
- Go to a record on the staff side /cgi-bin/koha/catalogue/detail.pl?biblionumber=1234
- Click the "show" link next to "Elasticsearch record:"
- a 404 page appears in a pop-up window (see attached screenshot)

The page is also a 404 if I inspect the link and open it in a new tab. The URL looks like /cgi-bin/koha/catalogue/showelastic.pl?id=1234

We are on 24.05.05.
Comment 1 Eric Phetteplace 2025-01-16 17:32:21 UTC
I tested on ktd just now and this works. Is it possible it was fixed in between 24.05 and now? I searched for an existing bug but if there was one, I was unable to find it.
Comment 2 Katrin Fischer 2025-01-17 09:40:49 UTC
It might have been fixed by bug 37494.
Comment 3 Nick Clemens (kidclamp) 2025-02-05 13:42:57 UTC
Created attachment 177534 [details] [review]
Bug 38912: Add option to include '_doc' type in ES record view

This bug has been filed several times and seems to relate to the ES version,
the version of Search::Elasticsearch and maybe other factors.

Because the version of ES being used may vary, and such the necessary perl module version
needed we should add an easy toggle to allow this feature to work, including the type if needed
and not if not.

To test:
 1 - Apply patches, updatedatabase
 2 - Enable Elasticsearch in Koha
 3 - View a record details page
 4 - Click 'Elastic record: Show'
 5 - You should see the record
 6 - Toggle ElasticsearchIncludeDocType system preference
 7 - You get a 404
 8 - Toggle it back
 9 - Record shows
10 - sudo apt-get install libsearch-elasticsearch-perl=6.81-1-koha1
11 - Restart all
12 - View ES record - it fails!
13 - Toggle the syspref
14 - It works!
15 - Sign off
Comment 4 Michał 2025-02-05 14:53:11 UTC
Is there no way to vary this depending on detected ES version in use? Adding a syspref would kinda suck unless there’s no way to avoid it… I was under the impression it was mostly a matter of changes between ES versions at least.
Comment 5 Nick Clemens (kidclamp) 2025-02-12 13:32:16 UTC
(In reply to Michał from comment #4)
> Is there no way to vary this depending on detected ES version in use? Adding
> a syspref would kinda suck unless there’s no way to avoid it… I was under
> the impression it was mostly a matter of changes between ES versions at
> least.

I think there are several factors: The ES perl module version, the ES version, and the state of the indexes, and possibly others.

The syspref makes it simple enough for the end user to change without a lot of overhead and I think it's a reasonable solution
Comment 6 Fridolin Somers 2025-02-14 15:22:30 UTC
We already did something like this with system preference "ElasticsearchCrossFields"
Comment 7 David Cook 2025-02-16 23:43:34 UTC
I thought the "type" parameter was dropped in ES 7 and ES 6 was EOL a few years ago?
Comment 8 David Cook 2025-02-16 23:44:13 UTC
I think there's a lot of other places in Koha that explode if using ES 6, no?
Comment 9 Katrin Fischer 2025-02-17 10:28:48 UTC
(In reply to David Cook from comment #8)
> I think there's a lot of other places in Koha that explode if using ES 6, no?

Support was dropped after 22.11:
https://wiki.koha-community.org/wiki/System_requirements_and_recommendations
Comment 10 David Cook 2025-02-17 23:31:04 UTC
(In reply to Katrin Fischer from comment #9)
> (In reply to David Cook from comment #8)
> > I think there's a lot of other places in Koha that explode if using ES 6, no?
> 
> Support was dropped after 22.11:
> https://wiki.koha-community.org/wiki/System_requirements_and_recommendations

So why are we looking at a patch for 24.05+?
Comment 11 Fridolin Somers 2025-02-20 14:13:52 UTC
For me this looks like perl deps have an impact :
I see we use Search::Elasticsearch 6.80 :
https://git.koha-community.org/Koha-community/Koha/src/branch/main/cpanfile#L102
https://debian.koha-community.org/koha/dists/24.11/main/binary-amd64/Packages

Shouldn't we upgrade to >= 7 ?
Debian Bookworm provides version 8.
https://packages.debian.org/bookworm/libsearch-elasticsearch-perl

Our installs on Ubuntu noble use perl dep with version 8.
Maybe this explains the difference of behavior.
Comment 12 David Cook 2025-02-20 23:07:35 UTC
> https://debian.koha-community.org/koha/dists/24.11/main/binary-amd64/Packages
(In reply to Fridolin Somers from comment #11)
> For me this looks like perl deps have an impact :
> I see we use Search::Elasticsearch 6.80 :
> https://git.koha-community.org/Koha-community/Koha/src/branch/main/
> cpanfile#L102
> https://debian.koha-community.org/koha/dists/24.11/main/binary-amd64/Packages
> 
> Shouldn't we upgrade to >= 7 ?
> Debian Bookworm provides version 8.
> https://packages.debian.org/bookworm/libsearch-elasticsearch-perl
> 
> Our installs on Ubuntu noble use perl dep with version 8.
> Maybe this explains the difference of behavior.

So I think that cpanfile line is an error/bug (which doesn't get carried over into debian/control fortunately).

In practice, most people should be using a newer version of libsearch-elasticsearch-perl if it's available on their system.
Comment 13 David Cook 2025-02-20 23:07:59 UTC
Sorry but I'm putting to FQA as I'm sure this isn't something we really want to do.
Comment 14 Magnus Enger 2025-03-10 13:40:15 UTC
I think I am running into this problem in the wild now. I have one server with this problem, running on Ubuntu 20.04 and this version of Search::Elasticsearch: 

$ sudo apt-cache policy libsearch-elasticsearch-perl
libsearch-elasticsearch-perl:
  Installed: 6.81-1-koha1
  Candidate: 6.81-1-koha1
  Version table:
 *** 6.81-1-koha1 500
        500 http://debian.koha-community.org/koha 23.11/main amd64 Packages
        100 /var/lib/dpkg/status
     6.00-1 500
        500 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages

I have another server without the problem, running Ubuntu 24.04 and this: 

$ sudo apt-cache policy libsearch-elasticsearch-perl
libsearch-elasticsearch-perl:
  Installed: 8.12-1
  Candidate: 8.12-1
  Version table:
 *** 8.12-1 500
        500 http://se.archive.ubuntu.com/ubuntu noble/universe amd64 Packages
        100 /var/lib/dpkg/status
     6.81-1-koha1 500
        500 http://debian.koha-community.org/koha 23.11/main amd64 Packages

I can't prove that the version of Search::Elasticsearch is what is causing this, but it is the only significant difference I have been able to find.
Comment 15 David Cook 2025-03-10 23:17:31 UTC
(In reply to Magnus Enger from comment #14)
> I think I am running into this problem in the wild now. I have one server
> with this problem, running on Ubuntu 20.04 and this version of
> Search::Elasticsearch: 

> I can't prove that the version of Search::Elasticsearch is what is causing
> this, but it is the only significant difference I have been able to find.

Looking at the Ubuntu website, it looks like Ubuntu 20.04 uses version 6 while 22.04 uses version 7 and all other newer Ubuntu versions use version 8 of the library.

Ubuntu 20.04 is EOL pretty soon, so should probably upgrade off that anyway. 

Plus, according to the wiki, Koha newer than 22.11 isn't supported on Ubuntu 20.04.
Comment 16 Magnus Enger 2025-03-11 07:47:26 UTC
(In reply to David Cook from comment #15)
> Plus, according to the wiki, Koha newer than 22.11 isn't supported on Ubuntu
> 20.04.

Yeah, I figured that out after a while. :-) 

Feels like this bug is a "wontfix", but maybe we should leave it open for others with the same problem?
Comment 17 Michał 2025-03-11 23:02:35 UTC
(In reply to Magnus Enger from comment #16)
> Feels like this bug is a "wontfix", but maybe we should leave it open for
> others with the same problem?

I don't think there's a point in cluttering opened bugs forever, it'd be searchable in exact same way no matter if opened or not for such people.

Unless the actionable solution for the bug would be deemed a patch to .deb control files to enforce a newer version, but the only practical result will be that the whole attempt to upgrade Koha on ancient Ubuntu/Debian versions would explode rather than just some small subset of features, so not sure if that'd really be an improvement either. But at least the system administrator would then know something's up :D
Comment 18 Fridolin Somers 2025-03-12 09:57:08 UTC
Shouldn't we remove "libsearch-elasticsearch-perl" from Koha packages repo ?