Bug 10935

Summary: CSV for basket and basketgroup are broken when translated
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: I18N/L10NAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: major    
Priority: P5 - low CC: f.demians, gitbot, gmcharlt, katrin.fischer, tomascohen
Version: master   
Hardware: All   
OS: All   
See Also: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10931
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7298
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9189
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 10935: CSV are broken using other language than english.
[SIGNED OFF] Bug 10935: CSV are broken using other language than english.
Bug 10935: Follow-up - adding a line break to fix first list item
Bug 10935: Follow-up - adding a line break to fix first list item
Bug 10935: CSV are broken using other language than english.
Bug 10935: Follow-up - adding a line break to fix first list item

Description Jonathan Druart 2013-09-23 15:05:23 UTC

    
Comment 1 Jonathan Druart 2013-09-23 15:18:30 UTC Comment hidden (obsolete)
Comment 2 Katrin Fischer 2013-09-23 21:21:24 UTC Comment hidden (obsolete)
Comment 3 Katrin Fischer 2013-09-23 21:21:48 UTC Comment hidden (obsolete)
Comment 4 Katrin Fischer 2013-09-23 21:22:18 UTC
Hi Jonathan, can you please check my follow-up and sign off?
Comment 5 Jonathan Druart 2013-09-24 08:11:15 UTC
(In reply to Katrin Fischer from comment #4)
> Hi Jonathan, can you please check my follow-up and sign off?

Katrin,
Yes! I started on the same way but it did not work. An empty line existed between the headers and the first row.
In fact, if there is no entry in the po file, it does not work. But if an entry exists, your patch fixes the issue.
Thanks for testing.
Comment 6 Jonathan Druart 2013-09-24 08:11:47 UTC Comment hidden (obsolete)
Comment 7 Chris Cormack 2013-09-24 08:43:35 UTC
Created attachment 21400 [details] [review]
Bug 10935: CSV are broken using other language than english.

To reproduce:
- cd misc/translator
- ./translate update LANG
- ./translate install LANG
- go to the Koha mainpage and change the language.
- go to acqui/basketgroup.pl?booksellerid=XX and try to export a
  basketgroup.
The headers is followed by the first basketgroup information. There is
no carriage return.

It looks like it is caused by a routine used by the translator script
(TmplTokenizer::string_canon).

To test this patch:
- apply it
- cd misc/translator
- ./translate -f update LANG
- translate headers in your po file
- ./translate -f install LANG
- go to acqui/basketgroup.pl?booksellerid=XX and try to export a
  basketgroup.
- verify that the csv looks good now.
- same thing for basket.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Good idea and seems to work - just fixing a small glitch
with the first entry of the list in a follow-up.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Comment 8 Chris Cormack 2013-09-24 08:45:36 UTC
Created attachment 21401 [details] [review]
Bug 10935: Follow-up - adding a line break to fix first list item

Without the line break after the include the first entry
of our list of orders was behind the header row in the first row
of the spreadsheet.

Adding the line break seems to fix that and translated CSV can
be exported correctly now.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Comment 9 Galen Charlton 2013-10-17 17:04:42 UTC
Pushed to master.  Thanks, Jonathan!
Comment 10 Tomás Cohen Arazi 2013-12-03 18:09:19 UTC
This patch has been pushed to 3.12.x, will be in 3.12.8.

Thanks Jonathan!
Comment 11 Katrin Fischer 2014-05-18 19:28:46 UTC
It looks like this bug has made a new appearance - I am seeing the problem again on master and 3.16 stable.
Comment 12 Katrin Fischer 2014-05-18 19:52:19 UTC
I think the problem again is a missing line break, but I don't know how to fix it.
Comment 13 Katrin Fischer 2014-05-18 20:01:22 UTC
Turns out, I was looking at the wrong installation. 

3.14: All information appears on one single line for the translated version, the English CSV is ok.

Master: We have way too many line breaks: 2 blank lines between the header row and the first line and 1 blank line after each order line?
Comment 14 Katrin Fischer 2014-05-18 20:13:31 UTC
Actually... after fixing some mess here it works wonderfully on master and I can only reproduce it on 3.14.6. 

Sorry for causing so much confusion.
Comment 15 Katrin Fischer 2014-05-18 20:32:20 UTC
Could someone do me a favor and retest this? Because I think I am going a bit crazy here... first it was 2 different installations I was looking at, now I missed to undo a change to the file I had done to see if that would fix the problem.

But I am pretty sure now that it's really broken in 3.14 and on master.
Comment 16 Katrin Fischer 2014-05-21 09:01:43 UTC
Tested in braod daylight... the bug can no longer be found on current master.
Comment 17 Jonathan Druart 2014-05-27 14:55:26 UTC
I reproduced on 3.14.x...
Comment 18 Jonathan Druart 2014-05-27 15:06:03 UTC
Note: The problem still exist on 3.14.x, even with these 2 patches.
The 2 files modules/acqui/csv/basket.tt and includes/csv_headers/acqui/basket.tt are the same on 3.14.x and on master.

BUT if they are translated, on master a blank line exist between headers and the first data row, on 3.14.x, they are on the same line.

I am reopening it for 3.14.x

Note that I didn't find the problem, if somebody want to have a look...
Comment 19 Jonathan Druart 2014-05-28 09:13:33 UTC
Good news! It seems to be fixed in 3.14.7. I used git-bisect and the following commit fixes the problem:

c0dd6bf598da1214d279977286762611eeff372b is the first bad commit
commit c0dd6bf598da1214d279977286762611eeff372b
Author: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Date:   Mon May 26 05:43:05 2014 -0300

    Translation updates for 3.14.07 release

:040000 040000 36759bb1e8c5b2904e5925852c2ad616f8672a40 7b0369787a19884a9dc84de400dc0fba44639870 M      misc