Bug 28234

Summary: TestBuilder->build_sample_biblio does not deal correctly with encoding
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: Test SuiteAssignee: Tomás Cohen Arazi (tcohen) <tomascohen>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: andrew, fridolin.somers, joonas.kylmala, kyle, martin.renvoize, tomascohen, victor
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28230
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
21.05.00,20.11.06,20.05.12
Circulation function:
Attachments: Bug 28234: Fix encoding issues in build_sample_biblio
[ALTERNATE] Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record
Bug 28234: Regression tests
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record
Bug 28234: Regression tests
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

Description Jonathan Druart 2021-04-27 08:45:41 UTC 

    


    


      



        
          Comment 1
        

        
          Jonathan Druart

        

        
        

        
          2021-04-27 08:47:13 UTC
        

      






Created attachment 120205 [details] [review]
Bug 28234: Fix encoding issues in build_sample_biblio

Found when working on bug 28230.
The tests generated a warning
 no mapping found for [0x2764] at position 4 in hell❤️ g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308.

The MARC::Record built by TestBuilder did not set the leader and unicode
flag

    


    


      



        
          Comment 2
        

        
          Tomás Cohen Arazi (tcohen)

        

        
        

        
          2021-04-27 11:25:27 UTC
        

      






Created attachment 120212 [details] [review]
[ALTERNATE] Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

    


    


      



        
          Comment 3
        

        
          Tomás Cohen Arazi (tcohen)

        

        
        

        
          2021-04-27 12:53:11 UTC
        

      






Created attachment 120214 [details] [review]
Bug 28234: Regression tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

    


    


      



        
          Comment 4
        

        
          Tomás Cohen Arazi (tcohen)

        

        
        

        
          2021-04-27 12:53:18 UTC
        

      






Created attachment 120215 [details] [review]
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

This patch makes the build_sample_biblio method, correctly set the UTF-8
flag for the MARC::Record object.

Tests are added.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/TestBuilder.t
FAIL: Tests fail! An unexpected encoding warning shows
3. Apply this patch
4. Repeat 2
=> SUCCESS: No warning! Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

    


    


      



        
          Comment 5
        

        
          Martin Renvoize (ashimema)

        

        
        

        
          2021-04-27 13:35:29 UTC
        

      






Created attachment 120225 [details] [review]
Bug 28234: Regression tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

    


    


      



        
          Comment 6
        

        
          Martin Renvoize (ashimema)

        

        
        

        
          2021-04-27 13:35:33 UTC
        

      






Created attachment 120226 [details] [review]
Bug 28234: TestBuilder->build_sample_biblio should set UTF-8 encoding for record

This patch makes the build_sample_biblio method, correctly set the UTF-8
flag for the MARC::Record object.

Tests are added.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/TestBuilder.t
FAIL: Tests fail! An unexpected encoding warning shows
3. Apply this patch
4. Repeat 2
=> SUCCESS: No warning! Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

    


    


      



        
          Comment 7
        

        
          Martin Renvoize (ashimema)

        

        
        

        
          2021-04-27 13:36:07 UTC
        

      






Good catch, works as expected. Signing off

    


    


      



        
          Comment 8
        

        
          Martin Renvoize (ashimema)

        

        
        

        
          2021-04-27 14:16:43 UTC
        

      






I think we can safely go straight for QA on this one..

    


    


      



        
          Comment 9
        

        
          Jonathan Druart

        

        
        

        
          2021-04-28 09:17:32 UTC
        

      






Pushed to master for 21.05, thanks to everybody involved!

    


    


      



        
          Comment 10
        

        
          Fridolin Somers

        

        
        

        
          2021-04-30 09:04:19 UTC
        

      






Pushed to 20.11.x for 20.11.06

    


    


      



        
          Comment 11
        

        
          Andrew Fuerste-Henry

        

        
        

        
          2021-05-24 18:56:02 UTC
        

      






Pushed to 20.05.x for 20.05.12

    


    


      



        
          Comment 12
        

        
          Victor Grousset/tuxayo

        

        
        

        
          2021-05-24 18:58:58 UTC
        

      






Not backported to oldoldstable (19.11.x). Feel free to ask if it's needed.