Bug 35141 - Prevent link_bibs_to_authorities from dying on search error
Summary: Prevent link_bibs_to_authorities from dying on search error
Status: Pushed to oldstable
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Nick Clemens (kidclamp)
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-10-24 11:50 UTC by Nick Clemens (kidclamp)
Modified: 2023-11-13 15:23 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.11.00,23.05.06,22.11.12


Attachments
Bug 35141: Catch ES search exceptions in link_bibs_to_authorities, warn, and continue (1.74 KB, patch)
2023-10-24 12:04 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 35141: Prevent link_bibs_to_authorities from dying on invalid metadata (1.25 KB, patch)
2023-10-24 12:04 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 35141: Catch ES search exceptions in link_bibs_to_authorities, warn, and continue (1.78 KB, patch)
2023-10-24 23:43 UTC, David Nind
Details | Diff | Splinter Review
Bug 35141: Prevent link_bibs_to_authorities from dying on invalid metadata (1.30 KB, patch)
2023-10-24 23:43 UTC, David Nind
Details | Diff | Splinter Review
Bug 35141: Catch ES search exceptions in link_bibs_to_authorities, warn, and continue (1.88 KB, patch)
2023-10-27 07:28 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 35141: Prevent link_bibs_to_authorities from dying on invalid metadata (1.40 KB, patch)
2023-10-27 07:28 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 35141: (QA follow-up) Increment the bad count (1.40 KB, patch)
2023-10-27 07:28 UTC, Marcel de Rooy
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2023-10-24 11:50:32 UTC
The script to link biblios to authorities can encounter search errors.

Currently the script simply dies on this error. It would be more useful to log the problem and continue the linking process.
Comment 1 Nick Clemens (kidclamp) 2023-10-24 12:04:11 UTC
Created attachment 157738 [details] [review]
Bug 35141: Catch ES search exceptions in link_bibs_to_authorities, warn, and continue

This patch simplay adds an eval and a warning to link_bibs_to_authorities to catch any errors while searching and continue linking.

To test:
1 - edit /etc/kohia/sites/kohadev/koha-conf.xml and add to elasticsearch stanza
    <request_timeout>0</request_timeout>
2 - perl misc/link_bibs_to_authorities.pl
3 - It dies immediately
4 - Apply patch
5 - perl misc/link_bibs_to_authorities.pl
6 - Now it tries every record, throwing warnings along the way, but completes
Comment 2 Nick Clemens (kidclamp) 2023-10-24 12:04:13 UTC
Created attachment 157739 [details] [review]
Bug 35141: Prevent link_bibs_to_authorities from dying on invalid metadata

When fetching the record we need to eval in case of exception

To test:
1 - Apply first patch
2 - perl misc/link_bibs_to_authorities.pl
3 - Note it dies on parser error at record 369
4 - Apply this patch
5 - perl misc/link_bibs_to_authorities.pl
6 - The process completes - with warning thrown on record 369
Comment 3 David Nind 2023-10-24 23:43:03 UTC
Created attachment 157766 [details] [review]
Bug 35141: Catch ES search exceptions in link_bibs_to_authorities, warn, and continue

This patch simplay adds an eval and a warning to link_bibs_to_authorities to catch any errors while searching and continue linking.

To test:
1 - edit /etc/kohia/sites/kohadev/koha-conf.xml and add to elasticsearch stanza
    <request_timeout>0</request_timeout>
2 - perl misc/link_bibs_to_authorities.pl
3 - It dies immediately
4 - Apply patch
5 - perl misc/link_bibs_to_authorities.pl
6 - Now it tries every record, throwing warnings along the way, but completes

Signed-off-by: David Nind <david@davidnind.com>
Comment 4 David Nind 2023-10-24 23:43:06 UTC
Created attachment 157767 [details] [review]
Bug 35141: Prevent link_bibs_to_authorities from dying on invalid metadata

When fetching the record we need to eval in case of exception

To test:
1 - Apply first patch
2 - perl misc/link_bibs_to_authorities.pl
3 - Note it dies on parser error at record 369
4 - Apply this patch
5 - perl misc/link_bibs_to_authorities.pl
6 - The process completes - with warning thrown on record 369

Signed-off-by: David Nind <david@davidnind.com>
Comment 5 Marcel de Rooy 2023-10-27 07:15:29 UTC
Filled Assignee :)
Comment 6 Marcel de Rooy 2023-10-27 07:21:27 UTC
Looking here
Comment 7 Marcel de Rooy 2023-10-27 07:28:07 UTC
Created attachment 157972 [details] [review]
Bug 35141: Catch ES search exceptions in link_bibs_to_authorities, warn, and continue

This patch simplay adds an eval and a warning to link_bibs_to_authorities to catch any errors while searching and continue linking.

To test:
1 - edit /etc/kohia/sites/kohadev/koha-conf.xml and add to elasticsearch stanza
    <request_timeout>0</request_timeout>
2 - perl misc/link_bibs_to_authorities.pl
3 - It dies immediately
4 - Apply patch
5 - perl misc/link_bibs_to_authorities.pl
6 - Now it tries every record, throwing warnings along the way, but completes

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 8 Marcel de Rooy 2023-10-27 07:28:09 UTC
Created attachment 157973 [details] [review]
Bug 35141: Prevent link_bibs_to_authorities from dying on invalid metadata

When fetching the record we need to eval in case of exception

To test:
1 - Apply first patch
2 - perl misc/link_bibs_to_authorities.pl
3 - Note it dies on parser error at record 369
4 - Apply this patch
5 - perl misc/link_bibs_to_authorities.pl
6 - The process completes - with warning thrown on record 369

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 9 Marcel de Rooy 2023-10-27 07:28:12 UTC
Created attachment 157974 [details] [review]
Bug 35141: (QA follow-up) Increment the bad count

Also move earlier print to warn for consistency.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 10 Tomás Cohen Arazi 2023-10-27 19:52:26 UTC
Pushed to master for 23.11.

Nice work everyone, thanks!
Comment 11 Fridolin Somers 2023-11-08 07:38:32 UTC
Pushed to 23.05.x for 23.05.06
Comment 12 Matt Blenkinsop 2023-11-13 15:23:28 UTC
Nice work everyone!

Pushed to oldstable for 22.11.x