Bug 22323

Summary: Cronjob runreport.pl has a CSV encoding issue
Product: Koha Reporter: Fridolin Somers <fridolin.somers>
Component: Command-line UtilitiesAssignee: Fridolin Somers <fridolin.somers>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: katrin.fischer, lucas, martin.renvoize, nick, robin
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15290
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on:    
Bug Blocks: 32012    
Attachments: Bug 22323: cronjob runreport.pl CSV add encoding
Bug 22323: cronjob runreport.pl CSV add encoding
Bug 22323: cronjob runreport.pl CSV add encoding

Description Fridolin Somers 2019-02-13 12:23:28 UTC
Cronjob runreport.pl runs SQL reports and can export as CSV.
This export needs to be UTF-8 encoded.
Comment 1 Fridolin Somers 2019-02-13 12:40:41 UTC
Created attachment 85061 [details] [review]
Bug 22323: cronjob runreport.pl CSV add encoding

Cronjob runreport.pl runs SQL reports and can export as CSV.
This export needs to be UTF-8 encoded.

This pathes replaces use of Text::CSV_XS by Text::CSV::Encoded like in other places like tools/viewlog.pl.
It adds a decode of headers bcause they will be encoded lika other lines.

Test plan:
1) Create a new item with itemnotes 'accentué'
2) Create a SQL report with :
SELECT barcode,itemnotes AS itè FROM items WHERE itemnotes LIKE 'accenté'
3) Run this report
4) You see well encoded header and content
5) Run from command line (replace X by report number) :
misc/cronjobs/runreport.pl X --format=csv --csv-header --store-results
6) You well encoded header and content
7) Go to saved reports table
8) Look at saved results of report
9) You well encoded header and content
Comment 2 Mikaël Olangcay Brisebois 2019-02-15 19:38:44 UTC
Created attachment 85189 [details] [review]
Bug 22323: cronjob runreport.pl CSV add encoding

Cronjob runreport.pl runs SQL reports and can export as CSV.
This export needs to be UTF-8 encoded.

This pathes replaces use of Text::CSV_XS by Text::CSV::Encoded like in other places like tools/viewlog.pl.
It adds a decode of headers bcause they will be encoded lika other lines.

Test plan:
1) Create a new item with itemnotes 'accentué'
2) Create a SQL report with :
SELECT barcode,itemnotes AS itè FROM items WHERE itemnotes LIKE 'accenté'
3) Run this report
4) You see well encoded header and content
5) Run from command line (replace X by report number) :
misc/cronjobs/runreport.pl X --format=csv --csv-header --store-results
6) You well encoded header and content
7) Go to saved reports table
8) Look at saved results of report
9) You well encoded header and content

Signed-off-by: Mikaël Olangcay Brisebois <mikael.olangcay-brisebois@inLibro.com>
Comment 3 Katrin Fischer 2019-02-17 11:22:57 UTC
Created attachment 85220 [details] [review]
Bug 22323: cronjob runreport.pl CSV add encoding

Cronjob runreport.pl runs SQL reports and can export as CSV.
This export needs to be UTF-8 encoded.

This pathes replaces use of Text::CSV_XS by Text::CSV::Encoded like in other places like tools/viewlog.pl.
It adds a decode of headers bcause they will be encoded lika other lines.

Test plan:
1) Create a new item with itemnotes 'accentué'
2) Create a SQL report with :
SELECT barcode,itemnotes AS itè FROM items WHERE itemnotes LIKE 'accenté'
3) Run this report
4) You see well encoded header and content
5) Run from command line (replace X by report number) :
misc/cronjobs/runreport.pl X --format=csv --csv-header --store-results
6) You well encoded header and content
7) Go to saved reports table
8) Look at saved results of report
9) You well encoded header and content

Signed-off-by: Mikaël Olangcay Brisebois <mikael.olangcay-brisebois@inLibro.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 4 Nick Clemens (kidclamp) 2019-02-19 14:03:43 UTC
Awesome work all!

Pushed to master for 19.05
Comment 5 Martin Renvoize 2019-02-25 15:32:29 UTC
Pushed to 18.11.x for 18.11.04
Comment 6 Lucas Gass 2019-03-06 16:48:10 UTC
backported to 18.05.x for 18.05.11
Comment 7 Fridolin Somers 2019-03-07 08:18:14 UTC
Pushed to 17.11.x for 17.11.17