Bug 28234 - TestBuilder->build_sample_biblio does not deal correctly with encoding
Summary: TestBuilder->build_sample_biblio does not deal correctly with encoding
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Tomás Cohen Arazi (tcohen)
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-04-27 08:45 UTC by Jonathan Druart
Modified: 2021-12-13 21:10 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
21.05.00,20.11.06,20.05.12
Circulation function:

Attachments
Bug 28234: Fix encoding issues in build_sample_biblio (1.33 KB, patch)
2021-04-27 08:47 UTC, Jonathan Druart 		Details | Diff | Splinter Review
[ALTERNATE] Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record (917 bytes, patch)
2021-04-27 11:25 UTC, Tomás Cohen Arazi (tcohen) 		Details | Diff | Splinter Review
Bug 28234: Regression tests (1.08 KB, patch)
2021-04-27 12:53 UTC, Tomás Cohen Arazi (tcohen) 		Details | Diff | Splinter Review
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record (1.25 KB, patch)
2021-04-27 12:53 UTC, Tomás Cohen Arazi (tcohen) 		Details | Diff | Splinter Review
Bug 28234: Regression tests (1.14 KB, patch)
2021-04-27 13:35 UTC, Martin Renvoize (ashimema) 		Details | Diff | Splinter Review
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record (1.31 KB, patch)
2021-04-27 13:35 UTC, Martin Renvoize (ashimema) 		Details | Diff | Splinter Review
Show Obsolete (4) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2021-04-27 08:45:41 UTC 

    


    


      



        
          Comment 1
        

        
          Jonathan Druart

        

        
        

        
          2021-04-27 08:47:13 UTC
        

      






Created attachment 120205 [details] [review]
Bug 28234: Fix encoding issues in build_sample_biblio

Found when working on bug 28230.
The tests generated a warning
 no mapping found for [0x2764] at position 4 in hell❤️ g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308.

The MARC::Record built by TestBuilder did not set the leader and unicode
flag

    


    


      



        
          Comment 2
        

        
          Tomás Cohen Arazi (tcohen)

        

        
        

        
          2021-04-27 11:25:27 UTC
        

      






Created attachment 120212 [details] [review]
[ALTERNATE] Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

    


    


      



        
          Comment 3
        

        
          Tomás Cohen Arazi (tcohen)

        

        
        

        
          2021-04-27 12:53:11 UTC
        

      






Created attachment 120214 [details] [review]
Bug 28234: Regression tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

    


    


      



        
          Comment 4
        

        
          Tomás Cohen Arazi (tcohen)

        

        
        

        
          2021-04-27 12:53:18 UTC
        

      






Created attachment 120215 [details] [review]
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

This patch makes the build_sample_biblio method, correctly set the UTF-8
flag for the MARC::Record object.

Tests are added.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/TestBuilder.t
FAIL: Tests fail! An unexpected encoding warning shows
3. Apply this patch
4. Repeat 2
=> SUCCESS: No warning! Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

    


    


      



        
          Comment 5
        

        
          Martin Renvoize (ashimema)

        

        
        

        
          2021-04-27 13:35:29 UTC
        

      






Created attachment 120225 [details] [review]
Bug 28234: Regression tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

    


    


      



        
          Comment 6
        

        
          Martin Renvoize (ashimema)

        

        
        

        
          2021-04-27 13:35:33 UTC
        

      






Created attachment 120226 [details] [review]
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

This patch makes the build_sample_biblio method, correctly set the UTF-8
flag for the MARC::Record object.

Tests are added.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/TestBuilder.t
FAIL: Tests fail! An unexpected encoding warning shows
3. Apply this patch
4. Repeat 2
=> SUCCESS: No warning! Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

    


    


      



        
          Comment 7
        

        
          Martin Renvoize (ashimema)

        

        
        

        
          2021-04-27 13:36:07 UTC
        

      






Good catch, works as expected. Signing off

    


    


      



        
          Comment 8
        

        
          Martin Renvoize (ashimema)

        

        
        

        
          2021-04-27 14:16:43 UTC
        

      






I think we can safely go straight for QA on this one..

    


    


      



        
          Comment 9
        

        
          Jonathan Druart

        

        
        

        
          2021-04-28 09:17:32 UTC
        

      






Pushed to master for 21.05, thanks to everybody involved!

    


    


      



        
          Comment 10
        

        
          Fridolin Somers

        

        
        

        
          2021-04-30 09:04:19 UTC
        

      






Pushed to 20.11.x for 20.11.06

    


    


      



        
          Comment 11
        

        
          Andrew Fuerste-Henry

        

        
        

        
          2021-05-24 18:56:02 UTC
        

      






Pushed to 20.05.x for 20.05.12

    


    


      



        
          Comment 12
        

        
          Victor Grousset/tuxayo

        

        
        

        
          2021-05-24 18:58:58 UTC
        

      






Not backported to oldoldstable (19.11.x). Feel free to ask if it's needed.