Bug 30402

Summary: Authority import hanging when replacing matched record
Product: Koha Reporter: Aleisha Amohia <aleisha>
Component: ToolsAssignee: Aleisha Amohia <aleisha>
Status: RESOLVED FIXED QA Contact: Testopia <testopia>
Severity: blocker    
Priority: P5 - low CC: andrewfh, jonathan.druart+koha, kyle
Version: masterKeywords: rel_21_11_candidate
Hardware: All   
OS: All   
Change sponsored?: Sponsored Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
22.05.00,21.11.05
Bug Depends on: 17600, 28758    
Bug Blocks:    
Attachments: Bug 30402: Import authorities subroutines to ImportBatch script
Bug 30402: Import authorities subroutines to ImportBatch script
Bug 30402: Import authorities subroutines to ImportBatch script

Description Aleisha Amohia 2022-03-29 22:48:23 UTC
We've discovered a bug with the MARC staged management.

When staging an authority record, we're applying a matching rule so that it should REPLACE a matched record. When trying to import, it hangs on 0% and never progresses or succeeds.

This does not happen with ignoring or adding, only replacing. This does not happen with biblios but does with authorities.
Comment 1 Aleisha Amohia 2022-03-29 23:41:35 UTC
Have tracked down this is due to authorities subroutines not being imported to the MARC management script.
Comment 2 Aleisha Amohia 2022-03-29 23:53:41 UTC
Created attachment 132494 [details] [review]
Bug 30402: Import authorities subroutines to ImportBatch script

The following authorities subroutines are used by the ImportBatch script
but are not accessible, because they aren't imported. This caused MARC
importing errors.
- GetAuthorityXML
- ModAuthority
- DelAuthority

These affected the BatchCommitRecords and BatchRevertRecords routines,
and it wasn't caught because there are no tests written for them.

To test:
1. Find an existing authority record, or import one to use.
2. Save this authority record (export/download).
3. Go to Admin -> Record matching rules. Create a new record matching
rule for authorities that matches on 001, Local-Number index.
4. Go to Tools -> Stage MARC for import. Upload the authority file you
just downloaded.
5. Change record type to authority.
6. Under 'Look for existing items in catalog?', set the record matching
rule to the rule you just made which matches on 001. If matching record
found, replace the existing one. If no match is found, ignore.
7. Stage the record. Once complete, a match should've been found.
8. Go to Staged MARC management.
9. Import the batch into the catalog. Notice it hangs and never
completes.

10. Apply the patch and restart services.
11. Repeat steps 4-9. This time importing should be successful.

Sponsored-by: Educational Services Australia SCIS
Comment 3 Owen Leonard 2022-04-04 13:30:34 UTC
Created attachment 132938 [details] [review]
Bug 30402: Import authorities subroutines to ImportBatch script

The following authorities subroutines are used by the ImportBatch
script but are not accessible, because they aren't imported. This
caused MARC importing errors.
- GetAuthorityXML
- ModAuthority
- DelAuthority

These affected the BatchCommitRecords and BatchRevertRecords routines,
and it wasn't caught because there are no tests written for them.

To test:
1. Find an existing authority record, or import one to use.
2. Save this authority record (export/download).
3. Go to Admin -> Record matching rules. Create a new record matching
   rule for authorities that matches on 001, Local-Number index.
4. Go to Tools -> Stage MARC for import. Upload the authority file you
   just downloaded.
5. Change record type to authority.
6. Under 'Look for existing items in catalog?', set the record matching
   rule to the rule you just made which matches on 001. If matching
   record found, replace the existing one. If no match is found, ignore.
7. Stage the record. Once complete, a match should've been found.
8. Go to Staged MARC management.
9. Import the batch into the catalog. Notice it hangs and never
   completes.

10. Apply the patch and restart services.
11. Repeat steps 4-9. This time importing should be successful.

Sponsored-by: Educational Services Australia SCIS

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Comment 4 Jonathan Druart 2022-04-06 14:28:23 UTC
Created attachment 133038 [details] [review]
Bug 30402: Import authorities subroutines to ImportBatch script

The following authorities subroutines are used by the ImportBatch
script but are not accessible, because they aren't imported. This
caused MARC importing errors.
- GetAuthorityXML
- ModAuthority
- DelAuthority

These affected the BatchCommitRecords and BatchRevertRecords routines,
and it wasn't caught because there are no tests written for them.

To test:
1. Find an existing authority record, or import one to use.
2. Save this authority record (export/download).
3. Go to Admin -> Record matching rules. Create a new record matching
   rule for authorities that matches on 001, Local-Number index.
4. Go to Tools -> Stage MARC for import. Upload the authority file you
   just downloaded.
5. Change record type to authority.
6. Under 'Look for existing items in catalog?', set the record matching
   rule to the rule you just made which matches on 001. If matching
   record found, replace the existing one. If no match is found, ignore.
7. Stage the record. Once complete, a match should've been found.
8. Go to Staged MARC management.
9. Import the batch into the catalog. Notice it hangs and never
   completes.

10. Apply the patch and restart services.
11. Repeat steps 4-9. This time importing should be successful.

Sponsored-by: Educational Services Australia SCIS

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 5 Fridolin Somers 2022-04-08 13:53:44 UTC
Pushed to master for 22.05, thanks to everybody involved 🦄
Comment 6 Kyle M Hall 2022-04-15 10:35:43 UTC
Pushed to 21.11.x for 21.11.05
Comment 7 Andrew Fuerste-Henry 2022-04-25 18:38:18 UTC
Missing dependency, not backported to 21.05