Bug 35618 - catalogue/showelastic.pl uses deprecated/removed parameter "type"
Summary: catalogue/showelastic.pl uses deprecated/removed parameter "type"
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low minor
Assignee: Michał
QA Contact: Julian Maurice
URL:
Keywords:
Depends on: 18829 33353
Blocks: 37494
  Show dependency treegraph
 
Reported: 2023-12-20 15:04 UTC by Michał
Modified: 2024-07-26 13:34 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
This fixes the display when clicking on "Show" for the "Elasticsearch record" entry for a record in the staff interface. Before this fix, a page not found (404) was displayed when viewing a record using Elasticsearch 7 or 8, or Open Search 1 ord 2. (Note that Elasticsearch 6 is no longer supported.)
Version(s) released in:
24.05.00,23.11.02
Circulation function:


Attachments
[PATCH] Bug 35618: Remove deprecated parameter "type" from showelastic.pl page (990 bytes, patch)
2023-12-20 15:13 UTC, Michał
Details | Diff | Splinter Review
Bug 35618: Remove deprecated parameter "type" from showelastic.pl page (1.00 KB, patch)
2023-12-21 21:28 UTC, David Nind
Details | Diff | Splinter Review
Bug 35618: Remove deprecated parameter "type" from showelastic.pl page (1.06 KB, patch)
2023-12-22 09:38 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 35618 - catalogue/showelastic.pl uses deprecated/removed parameter "type" [ES7.x] (1.08 KB, patch)
2024-07-01 05:27 UTC, Danyon Sewell
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Michał 2023-12-20 15:04:07 UTC
This can be treated as follow-up to Bug 33353. The showelastic.pl page still uses "type" parameter that was deprecated in ES7 and removed in ES8:
https://www.elastic.co/guide/en/elasticsearch/reference/6.0/removal-of-types.html

Bug 33353's patch already removed it in other places:
https://github.com/Koha-Community/Koha/commit/24aab6b9c52cb9846653d34d5630caf837e0508a

But one place remained:
https://github.com/Koha-Community/Koha/blob/d6314d32246a7e967e78f1427c338a3e13823617/catalogue/showelastic.pl#L63

This yields the following output in log:
```
[WARN] [Param] ** Unknown param (type) in (get) request. , called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /usr/share/koha/intranet/cgi-bin/catalogue/showelastic.pl line 61.
```
An exception is caught and user is redirected to 404 error page for all requests.

I will be created a patch for this in a moment, as I verified removing this field fixes the bug.
Comment 1 Michał 2023-12-20 15:13:34 UTC
Created attachment 160123 [details] [review]
[PATCH] Bug 35618: Remove deprecated parameter "type" from showelastic.pl page

This fixes the issue with ES8, should still work with ES7 as there types were just deprecated (and not fully removed yet). Won't work with ES6, but Koha master doesn't support it anymore already.

Should be backported to 23.11, but not to 23.05 or earlier.

To test, one should view a biblio record in staff interface and press "Show" next to "Elasticsearch Record:". With ES8 it will return 404 before applying this patch, check log in /var/log/koha/biblioteka/plack-intranet-error.log
Comment 2 David Nind 2023-12-21 21:28:28 UTC
Created attachment 160221 [details] [review]
Bug 35618: Remove deprecated parameter "type" from showelastic.pl page

This is a follow-up to Bug 33353. Types are gone in ES8 and deprecated in ES7. Support for ES6 is already dropped in Koha master.

The parameter has to be removed, otherwise an exception is thrown with ES8 (which is listed as officially supported by Koha now).

Signed-off-by: David Nind <david@davidnind.com>
Comment 3 David Nind 2023-12-21 21:34:43 UTC
Testing notes (using koha-testing-docker (KTD)):

1. Start up KTD with Elasticsearch 8 (ktd --es8 up)

2. In the staff interface, search and view a record.

3. For "Elasticsearch Record", click on the "Show" link.
   ==> Pop up window says "An error has occured Error 404..."
   ==> In /var/log/koha/kohadev/plack-intranet-error.log there is a warning:
       
       ES8:
       [2023/12/21 20:40:33] [WARN] [Request] ** [http://es:9200]-[400] no handler found for uri [/koha_kohadev_biblios/data/46] and method [GET], called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /kohadevbox/koha/catalogue/showelastic.pl line 61. With vars: {'request' => {'body' => undef,'method' => 'GET','serialize' => 'std','path' => '/koha_kohadev_biblios/data/46','ignore' => [],'qs' => {}},'body' => {'error' => 'no handler found for uri [/koha_kohadev_biblios/data/46] and method [GET]'},'status_code' => 400}
       
       ES7:
       [2023/12/21 21:05:31] [WARN] [Missing] ** [http://es:9200]-[404] Not Found, called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /kohadevbox/koha/catalogue/showelastic.pl line 61. With vars: {'request' => {'ignore' => [],'qs' => {},'path' => '/koha_kohadev_biblios/data/32','method' => 'GET','serialize' => 'std','body' => undef},'status_code' => 404,'body' => {'_index' => 'koha_kohadev_biblios','_type' => 'data','found' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),'_id' => '32'}}
       
       OS1:
       [2023/12/21 21:15:14] [WARN] [Missing] ** [http://es:9200]-[404] Not Found, called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /kohadevbox/koha/catalogue/showelastic.pl line 61. With vars: {'request' => {'ignore' => [],'path' => '/koha_kohadev_biblios/data/32','method' => 'GET','qs' => {},'serialize' => 'std','body' => undef},'status_code' => 404,'body' => {'_type' => 'data','_id' => '32','found' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),'_index' => 'koha_kohadev_biblios'}}

       OS2:
       [2023/12/21 21:25:04] [WARN] [Request] ** [http://es:9200]-[400] no handler found for uri [/koha_kohadev_biblios/data/5] and method [GET], called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /kohadevbox/koha/catalogue/showelastic.pl line 61. With vars: {'body' => {'error' => 'no handler found for uri [/koha_kohadev_biblios/data/5] and method [GET]'},'status_code' => 400,'request' => {'body' => undef,'method' => 'GET','path' => '/koha_kohadev_biblios/data/5','qs' => {},'ignore' => [],'serialize' => 'std'}}

4. Apply the patch and restart_all.

5. Repeat steps 2-3:
   ==> The pop-up window should now show the Elasticsearch record and no warning in the log [1].

6. Repeat using Elasticsearch 7 (ktd --es7 up), OpenSearch 1 (ktd --os1 up)and OpenSearch 2 (ktd --es7 up).


[1] Note that there are two warnings in the log - these are generated when you view the record in the staff interface, so I'm assuming that these are unrelated to this bug:

[2023/12/21 20:49:44] [WARN] DBIx::Class::ResultSetColumn::new(): Attempting to retrieve non-unique column 'biblionumber' on a resultset containing one-to-many joins will return duplicate results. at /kohadevbox/koha/Koha/Objects.pm line 426

[2023/12/21 20:49:44] [WARN] DBIx::Class::ResultSet::_construct_results(): Unable to properly collapse has_many results in iterator mode due to order criteria - performed an eager cursor slurp underneath. Consider using ->all() instead at /kohadevbox/koha/Koha/Objects.pm line 317
Comment 4 Michał 2023-12-21 22:01:15 UTC
Yeah the other warnings are probably unrelated directly to this. Repeatedly opening the Elasticsearch record view only repeatedly generates the single warn about type param being invalid.
Comment 5 Julian Maurice 2023-12-22 09:38:46 UTC
Created attachment 160227 [details] [review]
Bug 35618: Remove deprecated parameter "type" from showelastic.pl page

This is a follow-up to Bug 33353. Types are gone in ES8 and deprecated
in ES7. Support for ES6 is already dropped in Koha master.

The parameter has to be removed, otherwise an exception is thrown with
ES8 (which is listed as officially supported by Koha now).

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Comment 6 Katrin Fischer 2023-12-28 11:17:17 UTC
Pushed for 24.05!

Well done everyone, thank you!
Comment 7 Fridolin Somers 2024-01-03 16:26:22 UTC
Pushed to 23.11.x for 23.11.02
Comment 8 Lucas Gass (lukeg) 2024-01-12 22:06:40 UTC
Missing 23.05.x dependencies. No backport.
Comment 9 Danyon Sewell 2024-07-01 05:27:43 UTC
Created attachment 168306 [details] [review]
Bug 35618 - catalogue/showelastic.pl uses deprecated/removed parameter "type" [ES7.x]

it appears despite "type" being depreciated it's still required in some cases.

[2024/07/01 14:24:52] [WARN] [Param] ** Missing required param (type) in (get) request. , called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /usr/share/koha/intranet/cgi-bin/catalogue/showelastic.pl line 61.

subbing it out with type '_doc' seems to do the trick
Comment 10 Nick Clemens (kidclamp) 2024-07-11 13:55:13 UTC
(In reply to Danyon Sewell from comment #9)
> Created attachment 168306 [details] [review] [review]
> Bug 35618 - catalogue/showelastic.pl uses deprecated/removed parameter
> "type" [ES7.x]
> 
> it appears despite "type" being depreciated it's still required in some
> cases.
> 
> [2024/07/01 14:24:52] [WARN] [Param] ** Missing required param (type) in
> (get) request. , called from sub
> Search::Elasticsearch::Role::Client::Direct::__ANON__ at
> /usr/share/koha/intranet/cgi-bin/catalogue/showelastic.pl line 61.
> 
> subbing it out with type '_doc' seems to do the trick

As far as I can tell this only works if Koha has not been reindexed and the 'type' / 'data' is stil defined in the indexes - in both ES7 and ES8 it breaks when I add "_doc"

If this is still an issue can you file anew bug Danyon?
Comment 11 Fridolin Somers 2024-07-26 13:34:40 UTC
I've opened Bug 37494 to propose your fix Danyon.
Please go and test it ;)
Works for me with ES 8.

This report is closed so I hide your patch.

I love this super-cool ES feature.