Description
Andreas Roussos
2020-03-08 18:32:57 UTC
Created attachment 100313 [details]
MARC preview failure
Created attachment 100314 [details]
MARC preview working OK
Created attachment 100315 [details]
test record with embedded items containing UTF-8 data
Created attachment 100320 [details] [review] Bug 24827: Specify UTF-8 as encoding in new_from_xml() The MARC preview of staged records fails when a staged record contains items with UTF-8 characters (see attached screenshots). That's because in EmbedItemsInImportBiblio() the call to MARC::Record->new_from_xml() does not explicitly specify 'UTF-8' as the optional encoding to use. This patch fixes that, and also standardises the encoding name used in direct calls to new_from_xml() to 'UTF-8' instead of 'utf8' or 'utf-8'. Test plan (in a MARC21 instance): 0) Stage for import the test record attached to this bug report (make sure that 'Check for embedded item record data?' is set to 'Yes'), then go to Home > Tools > Manage staged MARC records and click on the filename you've just staged. 1) In the 'Citation' column, click on the hyperlink to launch the MARC preview -- notice how it fails to produce any output apart from the modal heading. 2) Apply the patch, and restart Plack if necessary. 3) Re-launch the MARC preview: this time the output should include the whole record, including the embedded item data in field 952. 4) Run the unit tests to ensure nothing was broken inadvertently: $ prove t $ prove xt Created attachment 100410 [details] [review] Bug 24827: Specify UTF-8 as encoding in new_from_xml() The MARC preview of staged records fails when a staged record contains items with UTF-8 characters (see attached screenshots). That's because in EmbedItemsInImportBiblio() the call to MARC::Record->new_from_xml() does not explicitly specify 'UTF-8' as the optional encoding to use. This patch fixes that, and also standardises the encoding name used in direct calls to new_from_xml() to 'UTF-8' instead of 'utf8' or 'utf-8'. Test plan (in a MARC21 instance): 0) Stage for import the test record attached to this bug report (make sure that 'Check for embedded item record data?' is set to 'Yes'), then go to Home > Tools > Manage staged MARC records and click on the filename you've just staged. 1) In the 'Citation' column, click on the hyperlink to launch the MARC preview -- notice how it fails to produce any output apart from the modal heading. 2) Apply the patch, and restart Plack if necessary. 3) Re-launch the MARC preview: this time the output should include the whole record, including the embedded item data in field 952. 4) Run the unit tests to ensure nothing was broken inadvertently: $ prove t $ prove xt Signed-off-by: Owen Leonard <oleonard@myacpl.org> Andreas, I agree with the changes. But could split the patch into 2 parts: 1 for the bugfix, 1 for the enhancement. That way the backport will not cause conflict (only the bugfix could be backported). Switch back the status to "Signed Off" when done. Created attachment 100694 [details] [review] Bug 24827: Standardise on 'UTF-8' as the encoding name This patch standardises the encoding name used in direct calls to new_from_xml() to 'UTF-8' instead of 'utf8' or 'utf-8'. Created attachment 100695 [details] [review] Bug 24827: Specify UTF-8 as encoding in new_from_xml() The MARC preview of staged records fails when a staged record contains items with UTF-8 characters (see attached screenshots). That's because in EmbedItemsInImportBiblio() the call to MARC::Record->new_from_xml() does not explicitly specify 'UTF-8' as the optional encoding to use. This patch fixes that. Test plan (in a MARC21 instance): 0) Stage for import the test record attached to this bug report (make sure that 'Check for embedded item record data?' is set to 'Yes'), then go to Home > Tools > Manage staged MARC records and click on the filename you've just staged. 1) In the 'Citation' column, click on the hyperlink to launch the MARC preview -- notice how it fails to produce any output apart from the modal heading. 2) Apply the patch, and restart Plack if necessary. 3) Re-launch the MARC preview: this time the output should include the whole record, including the embedded item data in field 952. 4) Run the unit tests to ensure nothing was broken inadvertently: $ prove t $ prove xt Thanks for your feedback, Jonathan. I've split my initial patch in two, hopefully I got this right ;-) Setting Status to "Signed Off". Created attachment 100737 [details] [review] Bug 24827: Standardise on 'UTF-8' as the encoding name This patch standardises the encoding name used in direct calls to new_from_xml() to 'UTF-8' instead of 'utf8' or 'utf-8'. Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Created attachment 100738 [details] [review] Bug 24827: Specify UTF-8 as encoding in new_from_xml() The MARC preview of staged records fails when a staged record contains items with UTF-8 characters (see attached screenshots). That's because in EmbedItemsInImportBiblio() the call to MARC::Record->new_from_xml() does not explicitly specify 'UTF-8' as the optional encoding to use. This patch fixes that. Test plan (in a MARC21 instance): 0) Stage for import the test record attached to this bug report (make sure that 'Check for embedded item record data?' is set to 'Yes'), then go to Home > Tools > Manage staged MARC records and click on the filename you've just staged. 1) In the 'Citation' column, click on the hyperlink to launch the MARC preview -- notice how it fails to produce any output apart from the modal heading. 2) Apply the patch, and restart Plack if necessary. 3) Re-launch the MARC preview: this time the output should include the whole record, including the embedded item data in field 952. 4) Run the unit tests to ensure nothing was broken inadvertently: $ prove t $ prove xt Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Created attachment 100739 [details] [review] Bug 24827: Add a test Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Nice work everyone! Pushed to master for 20.05 backported to 19.11.x branch for 19.11.05 patchset doesnt apply clean to 19.05.x, please rebase if needed |