Bug 15290 - SQL reports encoding problem
Summary: SQL reports encoding problem
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Reports (show other bugs)
Version: master
Hardware: All All
: P1 - high major (vote)
Assignee: Tomás Cohen Arazi
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-12-03 13:26 UTC by Vitor Fernandes
Modified: 2019-06-27 09:24 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 15290: SQL reports headers encoding problem (1.64 KB, patch)
2015-12-03 14:29 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 15290: (follow-up)SQL reports headers encoding problem (1.89 KB, patch)
2015-12-03 19:18 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 15290: (follow-up)SQL reports headers encoding problem (2.01 KB, patch)
2015-12-03 19:23 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 15290: (follow-up)SQL reports headers encoding problem (2.14 KB, patch)
2015-12-07 15:57 UTC, Frédéric Demians
Details | Diff | Splinter Review
Bug 15290: SQL reports headers encoding problem (1.78 KB, patch)
2015-12-07 15:58 UTC, Frédéric Demians
Details | Diff | Splinter Review
Bug 15290: (follow-up)SQL reports headers encoding problem (2.21 KB, patch)
2015-12-08 15:47 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 15290: SQL reports headers encoding problem (1.85 KB, patch)
2015-12-08 15:47 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Vitor Fernandes 2015-12-03 13:26:07 UTC
The problem appears when running a SQL report or exporting a SQL report. The column header are in incorrect encoding.

Test plan:

1 - Add a new SQL report using the following SQL for example:

SELECT borrowers.cardnumber 'Número cartão', CONCAT (borrowers.firstname, ' ' , borrowers.surname) Nome, categories.description Grupo, borrowers.branchcode Biblioteca,borrowers.dateenrolled 'Data de inscrição', borrowers.dateexpiry 'Data limite'
FROM borrowers
INNER JOIN categories
ON borrowers.categorycode = categories.categorycode
ORDER BY borrowers.cardnumber ASC

2 - Run the report
3 - The column headers are incorrect.
4 - Export the report to CSV.
5 - The column headers on the file are incorrect.
Comment 1 Tomás Cohen Arazi 2015-12-03 14:29:27 UTC Comment hidden (obsolete)
Comment 2 Tomás Cohen Arazi 2015-12-03 14:32:10 UTC
(In reply to Tomás Cohen Arazi from comment #1)
> - Export the report
> => SUCCESS: Encoding correct

It doesn't fix the exported files. Followup needed.
Comment 3 Héctor Eduardo Castro Avalos 2015-12-03 14:51:47 UTC
(In reply to Tomás Cohen Arazi from comment #2)
> (In reply to Tomás Cohen Arazi from comment #1)
> > - Export the report
> > => SUCCESS: Encoding correct
> 
> It doesn't fix the exported files. Followup needed.

Just fail in comma separated in tab separated and open document doesn't fail.
Comment 4 Vitor Fernandes 2015-12-03 14:58:30 UTC
CSV fails.
Tab separated text fails.
Open Document spreadsheet it's fine.
Comment 5 Héctor Eduardo Castro Avalos 2015-12-03 16:18:34 UTC
(In reply to Vitor Fernandes from comment #4)
> CSV fails.
> Tab separated text fails.
> Open Document spreadsheet it's fine.

I'm working on it, I found some issues
Comment 6 Héctor Eduardo Castro Avalos 2015-12-03 19:18:01 UTC Comment hidden (obsolete)
Comment 7 Héctor Eduardo Castro Avalos 2015-12-03 19:23:37 UTC Comment hidden (obsolete)
Comment 8 Frédéric Demians 2015-12-07 15:57:38 UTC Comment hidden (obsolete)
Comment 9 Frédéric Demians 2015-12-07 15:58:50 UTC Comment hidden (obsolete)
Comment 10 Jonathan Druart 2015-12-08 15:47:38 UTC
Created attachment 45501 [details] [review]
Bug 15290: (follow-up)SQL reports headers encoding problem

Fix SQL row data and headers enconding problem

To test:
-Apply patches
-Follow previous test plan
-Notice the headers and row data is displayed in UTF-8
 without double encoding
-Save in Comma separated text, Tab seprated text, and Open Document.
-Verify in the three options the file exported is Ok

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 11 Jonathan Druart 2015-12-08 15:47:42 UTC
Created attachment 45502 [details] [review]
Bug 15290: SQL reports headers encoding problem

To test:

- Create an SQL report with column names containing UTF characters
- Run the report
=> FAIL: The column names are double encoded
- Export the report
=> FAIL: The column names are double encoded in the exported file
- Apply the patch
- Re-run the report (reload works)
=> SUCCESS: Encoding correct
- Export the report
=> SUCCESS: Encoding correct
- Sign off :-D

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 12 Jonathan Druart 2015-12-08 15:50:22 UTC
Linked to bug 11944.
Must be backported to 3.22.x and 3.20.x
Comment 13 Kyle M Hall 2015-12-11 15:20:16 UTC
Patches pushed to master!

Thanks Hector and Tomas!
Comment 14 Julian Maurice 2015-12-11 16:07:40 UTC
Patches pushed to 3.22.x, will be in 3.22.1
Comment 15 Frédéric Demians 2015-12-11 17:12:15 UTC
This patch has been pushed to 3.20.x, will be in 3.20.7.