Summary: | Scripts not running under plack can cause duplication of ES records | ||
---|---|---|---|
Product: | Koha | Reporter: | Nick Clemens (kidclamp) <nick> |
Component: | Searching - Elasticsearch | Assignee: | Nick Clemens (kidclamp) <nick> |
Status: | CLOSED FIXED | QA Contact: | |
Severity: | major | ||
Priority: | P5 - low | CC: | 1joynelson, andrew, aspencatteam, ephetteplace, glasklas, jonathan.druart, lucas, martin.renvoize, severine.queune |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
See Also: |
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23986 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24823 |
||
GIT URL: | Change sponsored?: | --- | |
Patch complexity: | Trivial patch | Documentation contact: | |
Documentation submission: | Text to go in the release notes: | ||
Version(s) released in: |
20.05.00, 19.11.06, 19.05.11
|
Circulation function: | |
Attachments: |
Bug 25342: Force ES id as string
Bug 25342: Force ES id as string Bug 25342: Unit test Bug 25342: Force ES id as string Bug 25342: Unit test |
Description
Nick Clemens (kidclamp)
2020-05-01 14:48:43 UTC
Created attachment 104108 [details] [review] Bug 25342: Force ES id as string To test: 1 - Be using ES 2 - Find a unique record by searchign for title or seomthing 3 - Edit the record 4 - Repeat search and confirm you are returned to the details page 5 - Go to Tools->MARC modification templates 6 - Add a template and define an action 7 - Go to Tools->Batch record modification 8 - Select Enter a list of record numbers 9 - Enter the biblionumber of the record you searched for 10 - Use the marc modification template you setup 11 - Complete the modification 12 - Repeat your search 13 - note you get two results, both pointing to same biblionumber 14 - Check the ES index, note that you have two entries, one with the biblionumber, and the other with #.0, e.g.: curl -XGET 'es:9200/koha_kohadev_biblios/data/14.0?pretty' curl -XGET 'es:9200/koha_kohadev_biblios/data/14?pretty' 15 - Apply patch 16 - Delete and rebuild ES index perl misc/search_tools/rebuild_elasticsearch.pl -d -b 17 - Repeat 2-12 18 - No duplication this time Created attachment 104175 [details] [review] Bug 25342: Force ES id as string To test: 1 - Be using ES 2 - Find a unique record by searchign for title or seomthing 3 - Edit the record 4 - Repeat search and confirm you are returned to the details page 5 - Go to Tools->MARC modification templates 6 - Add a template and define an action 7 - Go to Tools->Batch record modification 8 - Select Enter a list of record numbers 9 - Enter the biblionumber of the record you searched for 10 - Use the marc modification template you setup 11 - Complete the modification 12 - Repeat your search 13 - note you get two results, both pointing to same biblionumber 14 - Check the ES index, note that you have two entries, one with the biblionumber, and the other with #.0, e.g.: curl -XGET 'es:9200/koha_kohadev_biblios/data/14.0?pretty' curl -XGET 'es:9200/koha_kohadev_biblios/data/14?pretty' 15 - Apply patch 16 - Delete and rebuild ES index perl misc/search_tools/rebuild_elasticsearch.pl -d -b 17 - Repeat 2-12 18 - No duplication this time Signed-off-by: David Nind <david@davidnind.com> Nick, this sounds familiar to me, did not we already get that kind of error with ES (int that needed to be stringified). Still no chance to get tests? Good catch. I guess this fix can be reverted once https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24823 is merged since this seams to be a catmandu-issue. Wrong bug, sorry. Meant to comment on bug 23986, but applies here as well. I will fix the issue with the patch not applying correctly in bug 24823. Created attachment 104509 [details] [review] Bug 25342: Unit test (In reply to Jonathan Druart from comment #3) > Nick, this sounds familiar to me, did not we already get that kind of error > with ES (int that needed to be stringified). > > Still no chance to get tests? The test is likely to pass before this patch as well, the issue is in the sending of the request somewhere, but this proves that at least stringifying it within Koha still gives what we expect (In reply to Nick Clemens from comment #0) > We received reports of duplicated records after batch modifications. > > It seems that when passing the ID to elasticsearch it is parsed as a string > under Plack, but is parsed as a number under CGI Reading the bug title I undersetood command line script, but it will actually fix script called in CGI mode, right? (In reply to Jonathan Druart from comment #9) > (In reply to Nick Clemens from comment #0) > > We received reports of duplicated records after batch modifications. > > > > It seems that when passing the ID to elasticsearch it is parsed as a string > > under Plack, but is parsed as a number under CGI > > Reading the bug title I undersetood command line script, but it will > actually fix script called in CGI mode, right? Yes, scripts that are excluded from plack like batch modification of bibs and imports are affected and fixed by this Created attachment 104685 [details] [review] Bug 25342: Force ES id as string To test: 1 - Be using ES 2 - Find a unique record by searchign for title or seomthing 3 - Edit the record 4 - Repeat search and confirm you are returned to the details page 5 - Go to Tools->MARC modification templates 6 - Add a template and define an action 7 - Go to Tools->Batch record modification 8 - Select Enter a list of record numbers 9 - Enter the biblionumber of the record you searched for 10 - Use the marc modification template you setup 11 - Complete the modification 12 - Repeat your search 13 - note you get two results, both pointing to same biblionumber 14 - Check the ES index, note that you have two entries, one with the biblionumber, and the other with #.0, e.g.: curl -XGET 'es:9200/koha_kohadev_biblios/data/14.0?pretty' curl -XGET 'es:9200/koha_kohadev_biblios/data/14?pretty' 15 - Apply patch 16 - Delete and rebuild ES index perl misc/search_tools/rebuild_elasticsearch.pl -d -b 17 - Repeat 2-12 18 - No duplication this time Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Created attachment 104686 [details] [review] Bug 25342: Unit test Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Nice work everyone! Pushed to master for 20.05 Pushed to 19.11.x for 19.11.06 backported to 19.05.x for 19.05.11 |