Bug 34043

Summary: Improve translation of CSV header templates
Product: Koha Reporter: Owen Leonard <oleonard>
Component: TemplatesAssignee: Owen Leonard <oleonard>
Status: RESOLVED FIXED QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: caroline.cyr-la-rose, fridolin.somers, jonathan.druart
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33339
Change sponsored?: --- Patch complexity: String patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.11.00
Bug Depends on: 21156    
Bug Blocks: 34845    
Attachments: Bug 34043: Improve translation of CSV header templates
Bug 34043: Improve translation of CSV header templates
Bug 34043: Remove incorrect spaces

Description Owen Leonard 2023-06-16 18:49:51 UTC
Koha has several templates which are used to define the headers for CSV export files, but they are not structured well for easy translation. For example, this is how the translation tool presents csv_headers/catalogue/itemsearch.tt in the .po file:

#: koha-tmpl/intranet-tmpl/prog/en/includes/csv_headers/catalogue/itemsearch.tt:1
#, c-format
msgid "%s %s %s %s \"Title\" %s \"Publication date\" %s \"Publisher\" %s \"Collection\" %s \"Barcode\" %s \"Serial enumeration\" %s \"Call number\" %s \"Home library\" %s \"Current library\" %s \"Shelving location\" %s \"Item type\" %s \"Inventory number\" %s \"Not for loan status\" %s \"Lost status\" %s \"Withdrawn status\" %s \"Checkouts\" %s \"Due date\" %s "

These files can be restructured to make the translation process easier.
Comment 1 Owen Leonard 2023-06-22 12:06:38 UTC
Created attachment 152558 [details] [review]
Bug 34043: Improve translation of CSV header templates

This patch reformats serverl CSV header templates so that it's easier to
translate the English strings. The templates now use the [% t("  ") %]
construction to ensure that each string is managed separately.

To test, apply the patch and go to Acquisitions.

- Locate a vendor with a basket and view the basket.
  - Click the "Export as CSV" button. The CSV file you get should be
    well-formatted.
  - Go to basket groups for that vendor and click the "Closed" tab.
    - Click the "Export as CSV" button for one of the baskets. The CSV
      file should be correct.
- Go to Acquisitions -> Late orders.
  - Check some checkboxes in the list of late orders. Click "Export as
    CSV" at the bottom of the page. The CSV file should be correct.

- Go to the catalog's item search page.
  - Perform a search which will return results. On the result page,
    choose "Export all results to -> CSV" and check the resulting CSV
    file.

- Go to Reports -> Statistics Wizards -> Cash register
  - Select "To a file" under "Output" and submit. Check the resulting
    CSV file.
- Go to Reports -> Orders by funds.
  - Choose "To a file" under "Output" and submit. Check the resulting
    CSV file.
Comment 2 Katrin Fischer 2023-06-22 18:34:43 UTC
We need to make sure that this is also tested with a translation/other language, because we keep having problems with the translation tools and CSV (translation kills line breaks for example).
Comment 3 Caroline Cyr La Rose 2023-08-17 06:46:16 UTC
Created attachment 154483 [details] [review]
Bug 34043: Improve translation of CSV header templates

This patch reformats serverl CSV header templates so that it's easier to
translate the English strings. The templates now use the [% t("  ") %]
construction to ensure that each string is managed separately.

To test, apply the patch and go to Acquisitions.

- Locate a vendor with a basket and view the basket.
  - Click the "Export as CSV" button. The CSV file you get should be
    well-formatted.
  - Go to basket groups for that vendor and click the "Closed" tab.
    - Click the "Export as CSV" button for one of the baskets. The CSV
      file should be correct.
- Go to Acquisitions -> Late orders.
  - Check some checkboxes in the list of late orders. Click "Export as
    CSV" at the bottom of the page. The CSV file should be correct.

- Go to the catalog's item search page.
  - Perform a search which will return results. On the result page,
    choose "Export all results to -> CSV" and check the resulting CSV
    file.

- Go to Reports -> Statistics Wizards -> Cash register
  - Select "To a file" under "Output" and submit. Check the resulting
    CSV file.
- Go to Reports -> Orders by funds.
  - Choose "To a file" under "Output" and submit. Check the resulting
    CSV file.

Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Comment 4 Caroline Cyr La Rose 2023-08-17 06:46:58 UTC
I translated in fr-CA and it worked for me.
Comment 5 Jonathan Druart 2023-09-01 09:56:52 UTC
Tested the item search export: there are too many unnecessary spaces

"Title",    "Publication date",    "Publisher",    "Collection",    "Barcode",    "Serial enumeration",    "Call number",    "Home library",    "Current library",    "Shelving location",    "Item type",    "Inventory number",    "Not for loan status",    "Lost status",    "Withdrawn status",    "Checkouts",    "Due date"
Comment 6 Tomás Cohen Arazi 2023-09-01 14:44:38 UTC
Pushed to master for 23.11.

Nice work everyone, thanks!
Comment 7 Tomás Cohen Arazi 2023-09-04 12:35:15 UTC
Created attachment 155197 [details] [review]
Bug 34043: Remove incorrect spaces

This bug looked cool and safe, but tests highlighted that the
(introduced) newlines were translated into spaces, which is not correct
in the CSV format (i.e. q{"Column 1" , "Column 2"} is not really
correct).

Also, the double quotes were forcibly introduced (semi-correct) but the
tests weren't adjusted.

We should really stop using templates for generating CSV, and use a
library for the task instead of manually crafting them. But that's for
another bug report.

This patch:

* Removes extra spaces in TT-generated CSV headers
* Adjusts the tests to the new format introduced by this report

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 8 Jonathan Druart 2023-09-04 13:33:14 UTC
Sorry, I didn't mean to PQA here, I let a comment about a problem.

Additionally I think this breaks t/db_dependent/Acquisition/GetBasketAsCSV.t
Comment 9 Jonathan Druart 2023-09-04 13:37:12 UTC
(In reply to Jonathan Druart from comment #8)
> Sorry, I didn't mean to PQA here, I let a comment about a problem.
> 
> Additionally I think this breaks t/db_dependent/Acquisition/GetBasketAsCSV.t

As well as t/db_dependent/Acquisition/GetBasketGroupAsCSV.t
Comment 10 Jonathan Druart 2023-09-04 13:39:47 UTC
And... I was behind. Should be fixed with Tomas's follow-up!
Comment 11 Fridolin Somers 2023-09-08 05:59:29 UTC
Not backported to 23.05.x