|
Description
David Cook
2024-11-11 03:55:42 UTC
Created attachment 174318 [details] [review] Bug 38416: Failover to MARCXML if cannot roundtrip USMARC during indexing This change failsover to MARCXML from USMARC if there are any warnings generated by MARC::File::USMARC::decode when trying to roundtrip the record. Test plan: 0. Apply the patch 1. Setup your koha-testing-docker to use Elasticsearch 2. Create a new record with 15,000 characters in the 500$a field 3. Index that record (e.g. perl misc/search_tools/rebuild_elasticsearch.pl --biblios -v -v) 4. Note that a warning saying the following appears: "Warnings encountered while roundtripping a MARC record to/from USMARC. Failing over to MARCXML" 5. View the "Elasticsearch record" on the detail page and note that the marc_format is MARCXML 6. Perform a search for the record (the keyword should be something that brings up other results too) 7. Note that the record appears correctly in the search results Created attachment 174319 [details] [review] Bug 38416: Add unit tests If we did push bug 38270, it would be tempting to failover to MARCXML_COMPRESSED actually... Note: When rebuilding the koha-testing-docker Elasticsearch indexes, I noticed zero impact on indexing time by adding the roundtripping step. Created attachment 174389 [details] [review] Bug 38416: Tidy Created attachment 174459 [details] [review] Bug 38416: Failover to MARCXML if cannot roundtrip USMARC during indexing This change failsover to MARCXML from USMARC if there are any warnings generated by MARC::File::USMARC::decode when trying to roundtrip the record. Test plan: 0. Apply the patch 1. Setup your koha-testing-docker to use Elasticsearch 2. Create a new record with 15,000 characters in the 500$a field 3. Index that record (e.g. perl misc/search_tools/rebuild_elasticsearch.pl --biblios -v -v) 4. Note that a warning saying the following appears: "Warnings encountered while roundtripping a MARC record to/from USMARC. Failing over to MARCXML" 5. View the "Elasticsearch record" on the detail page and note that the marc_format is MARCXML 6. Perform a search for the record (the keyword should be something that brings up other results too) 7. Note that the record appears correctly in the search results Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Created attachment 174460 [details] [review] Bug 38416: Add unit tests Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Created attachment 174461 [details] [review] Bug 38416: Tidy Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Created attachment 174542 [details] [review] Bug 38416: Failover to MARCXML if cannot roundtrip USMARC during indexing This change failsover to MARCXML from USMARC if there are any warnings generated by MARC::File::USMARC::decode when trying to roundtrip the record. Test plan: 0. Apply the patch 1. Setup your koha-testing-docker to use Elasticsearch 2. Create a new record with 15,000 characters in the 500$a field 3. Index that record (e.g. perl misc/search_tools/rebuild_elasticsearch.pl --biblios -v -v) 4. Note that a warning saying the following appears: "Warnings encountered while roundtripping a MARC record to/from USMARC. Failing over to MARCXML" 5. View the "Elasticsearch record" on the detail page and note that the marc_format is MARCXML 6. Perform a search for the record (the keyword should be something that brings up other results too) 7. Note that the record appears correctly in the search results Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Created attachment 174543 [details] [review] Bug 38416: Add unit tests Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Created attachment 174544 [details] [review] Bug 38416: Tidy Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Pushed for 24.11! Well done everyone, thank you! |