Bug 28234 - TestBuilder->build_sample_biblio does not deal correctly with encoding
Summary: TestBuilder->build_sample_biblio does not deal correctly with encoding
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Tomás Cohen Arazi
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-04-27 08:45 UTC by Jonathan Druart
Modified: 2021-12-13 21:10 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
21.05.00,20.11.06,20.05.12


Attachments
Bug 28234: Fix encoding issues in build_sample_biblio (1.33 KB, patch)
2021-04-27 08:47 UTC, Jonathan Druart
Details | Diff | Splinter Review
[ALTERNATE] Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record (917 bytes, patch)
2021-04-27 11:25 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 28234: Regression tests (1.08 KB, patch)
2021-04-27 12:53 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record (1.25 KB, patch)
2021-04-27 12:53 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 28234: Regression tests (1.14 KB, patch)
2021-04-27 13:35 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record (1.31 KB, patch)
2021-04-27 13:35 UTC, Martin Renvoize
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2021-04-27 08:45:41 UTC

    
Comment 1 Jonathan Druart 2021-04-27 08:47:13 UTC
Created attachment 120205 [details] [review]
Bug 28234: Fix encoding issues in build_sample_biblio

Found when working on bug 28230.
The tests generated a warning
 no mapping found for [0x2764] at position 4 in hell❤️ g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308.

The MARC::Record built by TestBuilder did not set the leader and unicode
flag
Comment 2 Tomás Cohen Arazi 2021-04-27 11:25:27 UTC
Created attachment 120212 [details] [review]
[ALTERNATE] Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 3 Tomás Cohen Arazi 2021-04-27 12:53:11 UTC
Created attachment 120214 [details] [review]
Bug 28234: Regression tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 4 Tomás Cohen Arazi 2021-04-27 12:53:18 UTC
Created attachment 120215 [details] [review]
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

This patch makes the build_sample_biblio method, correctly set the UTF-8
flag for the MARC::Record object.

Tests are added.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/TestBuilder.t
FAIL: Tests fail! An unexpected encoding warning shows
3. Apply this patch
4. Repeat 2
=> SUCCESS: No warning! Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 5 Martin Renvoize 2021-04-27 13:35:29 UTC
Created attachment 120225 [details] [review]
Bug 28234: Regression tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 6 Martin Renvoize 2021-04-27 13:35:33 UTC
Created attachment 120226 [details] [review]
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

This patch makes the build_sample_biblio method, correctly set the UTF-8
flag for the MARC::Record object.

Tests are added.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/TestBuilder.t
FAIL: Tests fail! An unexpected encoding warning shows
3. Apply this patch
4. Repeat 2
=> SUCCESS: No warning! Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 7 Martin Renvoize 2021-04-27 13:36:07 UTC
Good catch, works as expected. Signing off
Comment 8 Martin Renvoize 2021-04-27 14:16:43 UTC
I think we can safely go straight for QA on this one..
Comment 9 Jonathan Druart 2021-04-28 09:17:32 UTC
Pushed to master for 21.05, thanks to everybody involved!
Comment 10 Fridolin Somers 2021-04-30 09:04:19 UTC
Pushed to 20.11.x for 20.11.06
Comment 11 Andrew Fuerste-Henry 2021-05-24 18:56:02 UTC
Pushed to 20.05.x for 20.05.12
Comment 12 Victor Grousset/tuxayo 2021-05-24 18:58:58 UTC
Not backported to oldoldstable (19.11.x). Feel free to ask if it's needed.