Bug 34043

Summary:	Improve translation of CSV header templates
Product:	Koha	Reporter:	Owen Leonard <oleonard>
Component:	Templates	Assignee:	Owen Leonard <oleonard>
Status:	CLOSED FIXED	QA Contact:	Testopia <testopia>
Severity:	enhancement
Priority:	P5 - low	CC:	caroline.cyr-la-rose, fridolin.somers, jonathan.druart
Version:	Main
Hardware:	All
OS:	All
See Also:	https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33339
Change sponsored?:	---	Patch complexity:	String patch
Documentation contact:		Documentation submission:
Text to go in the release notes:		Version(s) released in:	23.11.00
Circulation function:
Bug Depends on:	21156
Bug Blocks:	34845
Attachments:	Bug 34043: Improve translation of CSV header templates Bug 34043: Improve translation of CSV header templates Bug 34043: Remove incorrect spaces

Description Owen Leonard 2023-06-16 18:49:51 UTC

Koha has several templates which are used to define the headers for CSV export files, but they are not structured well for easy translation. For example, this is how the translation tool presents csv_headers/catalogue/itemsearch.tt in the .po file:

#: koha-tmpl/intranet-tmpl/prog/en/includes/csv_headers/catalogue/itemsearch.tt:1
#, c-format
msgid "%s %s %s %s \"Title\" %s \"Publication date\" %s \"Publisher\" %s \"Collection\" %s \"Barcode\" %s \"Serial enumeration\" %s \"Call number\" %s \"Home library\" %s \"Current library\" %s \"Shelving location\" %s \"Item type\" %s \"Inventory number\" %s \"Not for loan status\" %s \"Lost status\" %s \"Withdrawn status\" %s \"Checkouts\" %s \"Due date\" %s "

These files can be restructured to make the translation process easier.

Comment 1 Owen Leonard 2023-06-22 12:06:38 UTC

Created attachment 152558 [details] [review]
Bug 34043: Improve translation of CSV header templates

This patch reformats serverl CSV header templates so that it's easier to
translate the English strings. The templates now use the [% t("  ") %]
construction to ensure that each string is managed separately.

To test, apply the patch and go to Acquisitions.

- Locate a vendor with a basket and view the basket.
  - Click the "Export as CSV" button. The CSV file you get should be
    well-formatted.
  - Go to basket groups for that vendor and click the "Closed" tab.
    - Click the "Export as CSV" button for one of the baskets. The CSV
      file should be correct.
- Go to Acquisitions -> Late orders.
  - Check some checkboxes in the list of late orders. Click "Export as
    CSV" at the bottom of the page. The CSV file should be correct.

- Go to the catalog's item search page.
  - Perform a search which will return results. On the result page,
    choose "Export all results to -> CSV" and check the resulting CSV
    file.

- Go to Reports -> Statistics Wizards -> Cash register
  - Select "To a file" under "Output" and submit. Check the resulting
    CSV file.
- Go to Reports -> Orders by funds.
  - Choose "To a file" under "Output" and submit. Check the resulting
    CSV file.

Comment 2 Katrin Fischer 2023-06-22 18:34:43 UTC

We need to make sure that this is also tested with a translation/other language, because we keep having problems with the translation tools and CSV (translation kills line breaks for example).

Comment 3 Caroline Cyr La Rose 2023-08-17 06:46:16 UTC

Created attachment 154483 [details] [review]
Bug 34043: Improve translation of CSV header templates

This patch reformats serverl CSV header templates so that it's easier to
translate the English strings. The templates now use the [% t("  ") %]
construction to ensure that each string is managed separately.

To test, apply the patch and go to Acquisitions.

- Locate a vendor with a basket and view the basket.
  - Click the "Export as CSV" button. The CSV file you get should be
    well-formatted.
  - Go to basket groups for that vendor and click the "Closed" tab.
    - Click the "Export as CSV" button for one of the baskets. The CSV
      file should be correct.
- Go to Acquisitions -> Late orders.
  - Check some checkboxes in the list of late orders. Click "Export as
    CSV" at the bottom of the page. The CSV file should be correct.

- Go to the catalog's item search page.
  - Perform a search which will return results. On the result page,
    choose "Export all results to -> CSV" and check the resulting CSV
    file.

- Go to Reports -> Statistics Wizards -> Cash register
  - Select "To a file" under "Output" and submit. Check the resulting
    CSV file.
- Go to Reports -> Orders by funds.
  - Choose "To a file" under "Output" and submit. Check the resulting
    CSV file.

Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>

Comment 4 Caroline Cyr La Rose 2023-08-17 06:46:58 UTC

I translated in fr-CA and it worked for me.

Comment 5 Jonathan Druart 2023-09-01 09:56:52 UTC

Tested the item search export: there are too many unnecessary spaces

"Title",    "Publication date",    "Publisher",    "Collection",    "Barcode",    "Serial enumeration",    "Call number",    "Home library",    "Current library",    "Shelving location",    "Item type",    "Inventory number",    "Not for loan status",    "Lost status",    "Withdrawn status",    "Checkouts",    "Due date"

Comment 6 Tomás Cohen Arazi (tcohen) 2023-09-01 14:44:38 UTC

Pushed to master for 23.11.

Nice work everyone, thanks!

Comment 7 Tomás Cohen Arazi (tcohen) 2023-09-04 12:35:15 UTC

Created attachment 155197 [details] [review]
Bug 34043: Remove incorrect spaces

This bug looked cool and safe, but tests highlighted that the
(introduced) newlines were translated into spaces, which is not correct
in the CSV format (i.e. q{"Column 1" , "Column 2"} is not really
correct).

Also, the double quotes were forcibly introduced (semi-correct) but the
tests weren't adjusted.

We should really stop using templates for generating CSV, and use a
library for the task instead of manually crafting them. But that's for
another bug report.

This patch:

* Removes extra spaces in TT-generated CSV headers
* Adjusts the tests to the new format introduced by this report

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Comment 8 Jonathan Druart 2023-09-04 13:33:14 UTC

Sorry, I didn't mean to PQA here, I let a comment about a problem.

Additionally I think this breaks t/db_dependent/Acquisition/GetBasketAsCSV.t

Comment 9 Jonathan Druart 2023-09-04 13:37:12 UTC

(In reply to Jonathan Druart from comment #8)
> Sorry, I didn't mean to PQA here, I let a comment about a problem.
> 
> Additionally I think this breaks t/db_dependent/Acquisition/GetBasketAsCSV.t

As well as t/db_dependent/Acquisition/GetBasketGroupAsCSV.t

Comment 10 Jonathan Druart 2023-09-04 13:39:47 UTC

And... I was behind. Should be fixed with Tomas's follow-up!

Comment 11 Fridolin Somers 2023-09-08 05:59:29 UTC

Not backported to 23.05.x