Bug 28234

Summary: TestBuilder->build_sample_biblio does not deal correctly with encoding
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: Test SuiteAssignee: Tomás Cohen Arazi <tomascohen>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: andrewfh, fridolin.somers, joonas.kylmala, kyle, martin.renvoize, tomascohen, victor
Version: master   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28230
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
21.05.00,20.11.06,20.05.12
Attachments: Bug 28234: Fix encoding issues in build_sample_biblio
[ALTERNATE] Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record
Bug 28234: Regression tests
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record
Bug 28234: Regression tests
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

Description Jonathan Druart 2021-04-27 08:45:41 UTC

    
Comment 1 Jonathan Druart 2021-04-27 08:47:13 UTC
Created attachment 120205 [details] [review]
Bug 28234: Fix encoding issues in build_sample_biblio

Found when working on bug 28230.
The tests generated a warning
 no mapping found for [0x2764] at position 4 in hell❤️ g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308.

The MARC::Record built by TestBuilder did not set the leader and unicode
flag
Comment 2 Tomás Cohen Arazi 2021-04-27 11:25:27 UTC
Created attachment 120212 [details] [review]
[ALTERNATE] Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 3 Tomás Cohen Arazi 2021-04-27 12:53:11 UTC
Created attachment 120214 [details] [review]
Bug 28234: Regression tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 4 Tomás Cohen Arazi 2021-04-27 12:53:18 UTC
Created attachment 120215 [details] [review]
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

This patch makes the build_sample_biblio method, correctly set the UTF-8
flag for the MARC::Record object.

Tests are added.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/TestBuilder.t
FAIL: Tests fail! An unexpected encoding warning shows
3. Apply this patch
4. Repeat 2
=> SUCCESS: No warning! Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 5 Martin Renvoize 2021-04-27 13:35:29 UTC
Created attachment 120225 [details] [review]
Bug 28234: Regression tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 6 Martin Renvoize 2021-04-27 13:35:33 UTC
Created attachment 120226 [details] [review]
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

This patch makes the build_sample_biblio method, correctly set the UTF-8
flag for the MARC::Record object.

Tests are added.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/TestBuilder.t
FAIL: Tests fail! An unexpected encoding warning shows
3. Apply this patch
4. Repeat 2
=> SUCCESS: No warning! Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 7 Martin Renvoize 2021-04-27 13:36:07 UTC
Good catch, works as expected. Signing off
Comment 8 Martin Renvoize 2021-04-27 14:16:43 UTC
I think we can safely go straight for QA on this one..
Comment 9 Jonathan Druart 2021-04-28 09:17:32 UTC
Pushed to master for 21.05, thanks to everybody involved!
Comment 10 Fridolin Somers 2021-04-30 09:04:19 UTC
Pushed to 20.11.x for 20.11.06
Comment 11 Andrew Fuerste-Henry 2021-05-24 18:56:02 UTC
Pushed to 20.05.x for 20.05.12
Comment 12 Victor Grousset/tuxayo 2021-05-24 18:58:58 UTC
Not backported to oldoldstable (19.11.x). Feel free to ask if it's needed.