Bug 26543

Summary: Elasticsearch - Importing biblios with multiple items can cause incorrect index
Product: Koha Reporter: Nick Clemens <nick>
Component: Searching - ElasticsearchAssignee: Nick Clemens <nick>
Status: RESOLVED DUPLICATE QA Contact:
Severity: major    
Priority: P5 - low CC: andrewfh, jonathan.druart, katrin.fischer, m.de.rooy, severine.queune
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 25265, 26557, 26903    
Bug Blocks:    
Attachments: Bug 26543: Refactor import code to reindex once per bib
Bug 26543: Unit tests

Description Nick Clemens 2020-09-25 18:29:27 UTC
Similar to bug 25265 we see many request for indexing a single biblio with multiple items attached when importing and matching records

Additionally, the current code repeatedly generates an indexer object, we can reuse the single object to avoid creating multiple connections
Comment 1 Nick Clemens 2020-09-25 18:53:55 UTC
Created attachment 110771 [details] [review]
Bug 26543: Refactor import code to reindex once per bib

To test:
0 - Be using Elasticsearch, apply patches, reindex
1 - Add multiple items to several records in your catalog
2 - Export those records
3 - Update the items on the records in Koha to be 'damaged'
4 - Import the file exported above, matchign on biblionumber and overlaying items
5 - Perform a search to return the imported records
6 - Confirm all items have damaged correctly removed in search results
7 - Import without matching and ignoring items to confirm new records are added
8 - Search to return records and verify they are indexed (shoudl see 2 copies)
9 - Alter the barcodes of the items in koha
10 - Import the file matching on biblionumber and adding items
11 - Search tp return records, confirm the new items are indexed
12 - Export some authority records
13 - Import them without matching and confirm new records are searchable
14 - Alter the existing records, and match on AuthID, you will need to create a matching rule for this
15 - Import and match and confirm the records are searchable and reflect the change back to original values
Comment 2 Nick Clemens 2020-09-25 18:54:01 UTC
Created attachment 110772 [details] [review]
Bug 26543: Unit tests
Comment 3 Andrew Fuerste-Henry 2020-09-25 19:50:54 UTC
I got an error trying to apply this:

Applying: Bug 26543: Refactor import code to reindex once per bib
error: sha1 information is lacking or useless (C4/Biblio.pm).
error: could not build fake ancestor
Patch failed at 0001 Bug 26543: Refactor import code to reindex once per bib
Comment 4 Andrew Fuerste-Henry 2020-09-28 16:43:00 UTC
Applies now that 25265 is pushed!
Comment 5 Andrew Fuerste-Henry 2020-11-02 20:02:30 UTC
Updating test plan. Existing step 9 led to an error as one attempted to create items based on a 952 containing an existing itemnumber: 

To test:
0 - Be using Elasticsearch, apply patches, reindex
1 - Add multiple items to several records in your catalog
2 - Export those records
3 - Update the items on the records in Koha to be 'damaged'
4 - Import the file exported above, matchign on biblionumber and overlaying items
5 - Perform a search to return the imported records
6 - Confirm all items have damaged correctly removed in search results
7 - Import without matching and ignoring items to confirm new records are added
8 - Search to return records and verify they are indexed (shoudl see 2 copies)
9 - Alter exported records to remove itemnumber and barcode
10 - Import the file matching on biblionumber and adding items
11 - Search tp return records, confirm the new items are indexed
12 - Export some authority records
13 - Import them without matching and confirm new records are searchable
14 - Alter the existing records, and match on AuthID, you will need to create a matching rule for this
15 - Import and match and confirm the records are searchable and reflect the change back to original values
Comment 6 Andrew Fuerste-Henry 2020-11-02 20:03:13 UTC
Failing on step 13. New authorities are created but don't come up in searches.
Comment 7 Andrew Fuerste-Henry 2020-11-03 01:09:01 UTC
See blocking bug, 26903. Turns out the behavior I ran into happens on master without this patch.
Comment 8 Nick Clemens 2023-08-18 08:51:10 UTC

*** This bug has been marked as a duplicate of bug 30822 ***