Bug 41620

Summary: Make itemsearch.pl use Koha::CSV for CSV generation
Product: Koha Reporter: Tomás Cohen Arazi (tcohen) <tomascohen>
Component: Architecture, internals, and plumbingAssignee: Tomás Cohen Arazi (tcohen) <tomascohen>
Status: Needs Signoff --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: david, martin.renvoize, nick, tomascohen
Version: Main   
Hardware: All   
OS: All   
GIT URL: Initiative type: ---
Sponsorship status: --- Comma delimited list of Sponsors:
Crowdfunding goal: 0 Patch complexity: Medium patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Bug Depends on: 41617, 41619, 40880    
Bug Blocks: 41621    
Attachments: Bug 41620: Introduce Koha::CSV::ItemSearch
Bug 41620: Use Koha::CSV::ItemSearch for CSV exports

Description Tomás Cohen Arazi (tcohen) 2026-01-14 19:51:38 UTC
This will save us a lot of headaches for sure.
Comment 1 Tomás Cohen Arazi (tcohen) 2026-01-14 20:09:45 UTC
Created attachment 191467 [details] [review]
Bug 41620: Introduce Koha::CSV::ItemSearch

This patch adds Koha::CSV::ItemSearch, a specialized CSV generator for
item search results with predefined headers and formatting.

The class:
- Extends Koha::CSV with always_quote enabled for Excel compatibility
- Provides 21 predefined column headers for item search exports
- Formats Koha::Item objects into CSV-ready field arrays
- Handles authorized values, dates, and library names
- Supports MARC21 and UNIMARC title formatting

This replaces the template-based CSV generation in itemsearch.pl,
eliminating spacing issues and providing a testable, maintainable
approach to item search CSV exports.

Test plan:
1. Apply patch
2. Run:
   $ ktd --shell
  k$ prove t/db_dependent/Koha/CSV/ItemSearch.t
=> SUCCESS: Tests pass!
3. Tests cover:
   - Object initialization with always_quote enabled
   - Header generation (21 columns)
   - Item formatting with all field types
   - Authorized value lookups
   - Date formatting
   - Library name resolution
   - CSV output to filehandle
4. Sign off :-D
Comment 2 Tomás Cohen Arazi (tcohen) 2026-01-14 20:09:47 UTC
Created attachment 191468 [details] [review]
Bug 41620: Use Koha::CSV::ItemSearch for CSV exports

This patch replaces template-based CSV generation with Koha::CSV::ItemSearch
in both itemsearch.pl and item-export.pl, eliminating spacing issues around
delimiters that caused Excel to display quotes as literal characters.

Changes:
- Update itemsearch.pl to use Koha::CSV::ItemSearch
- Update item-export.pl to use Koha::CSV::ItemSearch
- Remove obsolete CSV templates
- Stream output directly without template overhead

This fixes the issue where extra spaces around commas in the templates
caused Excel to treat quotes as part of the data rather than CSV delimiters.

Test plan:
1. Apply patches
2. Test itemsearch.pl:
   a. Go to Cataloging > Item search
   b. Perform a search
   c. Click "Export results" > "CSV"
   d. Open items.csv in Excel
   => SUCCESS: Fields display without quotes
3. Test item-export.pl:
   a. Go to a bibliographic record
   b. Select multiple items
   c. Click "Export selected items"
   d. Open items.csv in Excel
   => SUCCESS: Fields display without quotes
4. Verify all 21 columns are present and correctly formatted
5. Verify dates, authorized values, and library names display correctly
6. Sign off :-D
Comment 3 David Nind 2026-01-16 06:34:22 UTC
Getting a sha1/fake ancestor error when applying the patch:

git bz apply 41620

Bug 41620 Depends on bug 41619 (Signed Off)
Follow? [(y)es, (n)o] y

Bug 41619 - Add `Koha::CSV`

191491 - Bug 41619: Add Koha::CSV

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 41619: Add Koha::CSV

Bug 41620 - Make itemsearch.pl use Koha::CSV for CSV generation

191467 - Bug 41620: Introduce Koha::CSV::ItemSearch
191468 - Bug 41620: Use Koha::CSV::ItemSearch for CSV exports

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 41620: Introduce Koha::CSV::ItemSearch
Applying: Bug 41620: Use Koha::CSV::ItemSearch for CSV exports
error: sha1 information is lacking or useless (koha-tmpl/intranet-tmpl/prog/en/includes/catalogue/itemsearch_item.csv.inc).
error: could not build fake ancestor
Patch failed at 0001 Bug 41620: Use Koha::CSV::ItemSearch for CSV exports
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-41620-Use-KohaCSVItemSearch-for-CSV-exports-4t4s7561.patch
Comment 4 Tomás Cohen Arazi (tcohen) 2026-01-23 14:16:34 UTC
Sorry, it was missing a dependency. It applies cleanly so far.
Comment 5 David Nind 2026-01-23 18:43:17 UTC
(In reply to Tomás Cohen Arazi (tcohen) from comment #4)
> Sorry, it was missing a dependency. It applies cleanly so far.

Thanks Tomás!
Comment 6 David Nind 2026-01-23 19:08:07 UTC
My observations from exporting - not sure if this is another issue, or what is expected.

Happy to sign off it this is okay.

Testing notes (using KTD):

1. CSV export and opening with a text editor:
   - All 21 columns exported
   - Comma separated, quotes around values
   - Items with accent characters: content is okay in the text file
   - Items with other characters: doesn't export correctly, for example record 233 where the title is "МОДЕРНИЗАЦИЯ НАЦИОНАЛЬНЫХ СИСТЕМ РАСПРОСТРАНЕНИЯ ЗНАНИЙ ПО ВОПРОСАМ СЕЛЬСКОГО ХОЗЯЙСТВА":
    " ÐÐÐÐРÐÐÐÐЊÐЯ ÐÐЊÐÐÐÐÐЬÐЫХ СÐСТÐРРÐСÐРÐСТРÐÐÐÐÐЯ ÐÐÐÐÐÐ ÐÐ ÐÐÐРÐСÐРСÐÐЬСÐÐÐРХÐÐЯÐСТÐÐ Qamar, M.K.","2008","FAO","Reference","39999000010114","513","","","Fairview","Fairview","General stacks","Books","","0","","","","2014-09-04","0","",""

2. CSV export and opening with LibreOffice:
   - All 21 columns exported
   - Items with accent characters: content is okay in the text file, but in LibreOffice doesn't show correctly, for example record 192 where the title is "Páipéir bhána agus páipéir bhreaca.":
     "P�ip�ir bh�na agus p�ip�ir bhreaca." � Cadhain, M�irt�n.
   - The item in the CSV that didn't show correcty, is shown correctly in LibreOffice: МОДЕРНИЗАЦИЯ НАЦИОНАЛЬНЫХ СИСТЕМ РАСПРОСТРАНЕНИЯ ЗНАНИЙ ПО ВОПРОСАМ СЕЛЬСКОГО ХОЗЯЙСТВА Qamar, M.K.

2. CSV export and opening with Excel (used the web-based version of Excel):
   - All 21 columns exported
   - For the barcode column, shows as 4E+13 (from the item search results) or 3.9999E+13 (record holding table export) - changing the column to a number displays the number correctly
   - Items with accent characters: shows correctly (which is the opposite of LIbreOffice!)
   - Items with other characters: don't show correctly, for the one above: МОДЕРÐИЗÐЦИЯ ÐÐЦИОÐÐЛЬÐЫХ СИСТЕМ Ð ÐСПРОСТРÐÐЕÐИЯ ЗÐÐÐИЙ ПО ВОПРОСÐМ СЕЛЬСКОГО ХОЗЯЙСТВРQamar, M.K.