Bug 24679 - Allow to apply a MARC modification template in export tool
Summary: Allow to apply a MARC modification template in export tool
Status: Signed Off
Alias: None
Product: Koha
Classification: Unclassified
Component: Tools (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Julian Maurice
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks: 25253
  Show dependency treegraph
 
Reported: 2020-02-18 11:48 UTC by Julian Maurice
Modified: 2026-01-27 09:58 UTC (History)
4 users (show)

See Also:
GIT URL:
Initiative type: ---
Sponsorship status: ---
Comma delimited list of Sponsors:
Crowdfunding goal: 0
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
Bug 24679: Allow to apply a MARC modification template in export tool (20.04 KB, patch)
2020-02-18 11:49 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 24679: Allow to apply a MARC modification template in export tool (20.16 KB, patch)
2020-02-20 09:29 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 24679: Allow to apply a MARC modification template in export tool (20.24 KB, patch)
2020-02-27 14:29 UTC, Devinim
Details | Diff | Splinter Review
Bug 24679: Allow to apply a MARC modification template in export tool (20.25 KB, patch)
2020-10-13 08:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 24679: Return the correct id in AddModificationTemplate (1.16 KB, patch)
2020-10-13 08:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 24679: Fix exporter tests (2.52 KB, patch)
2020-10-13 08:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 24679: Allow to apply a MARC modification template in export tool (14.82 KB, patch)
2025-09-10 23:53 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 24679: Return the correct id in AddModificationTemplate (1.18 KB, patch)
2025-09-10 23:53 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 24679: Fix exporter tests (6.81 KB, patch)
2025-09-10 23:53 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 24679: Allow to apply a MARC modification template in export tool (14.87 KB, patch)
2026-01-26 18:09 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 24679: Return the correct id in AddModificationTemplate (1.24 KB, patch)
2026-01-26 18:09 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 24679: Fix exporter tests (6.87 KB, patch)
2026-01-26 18:09 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 24679: Tidy (5.48 KB, patch)
2026-01-27 09:53 UTC, Julian Maurice
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Julian Maurice 2020-02-18 11:48:58 UTC

    
Comment 1 Julian Maurice 2020-02-18 11:49:46 UTC
Created attachment 99175 [details] [review]
Bug 24679: Allow to apply a MARC modification template in export tool

Test plan:
1. Create a MARC modification template with a simple action (for
   instance, "Add new field 999$9 with value foobar")
2. Go to Tools » Export data
3. Select some records (for instance, from biblionumber 1 to 10)
4. In options, select the MARC modification template you just created
5. Select File format 'MARC'
6. Click on 'Export bibliographic records'
7. Verify that exported records contain the 999$9 field
8. Repeat steps 3-7 with other export formats, and with authority
   records
Comment 2 Julian Maurice 2020-02-20 09:29:52 UTC
Created attachment 99304 [details] [review]
Bug 24679: Allow to apply a MARC modification template in export tool

Test plan:
1. Create a MARC modification template with a simple action (for
   instance, "Add new field 999$9 with value foobar")
2. Go to Tools » Export data
3. Select some records (for instance, from biblionumber 1 to 10)
4. In options, select the MARC modification template you just created
5. Select File format 'MARC'
6. Click on 'Export bibliographic records'
7. Verify that exported records contain the 999$9 field
8. Repeat steps 3-7 with other export formats, and with authority
   records
Comment 3 Devinim 2020-02-27 14:29:52 UTC
Created attachment 99700 [details] [review]
Bug 24679: Allow to apply a MARC modification template in export tool

Test plan:
1. Create a MARC modification template with a simple action (for
   instance, "Add new field 999$9 with value foobar")
2. Go to Tools » Export data
3. Select some records (for instance, from biblionumber 1 to 10)
4. In options, select the MARC modification template you just created
5. Select File format 'MARC'
6. Click on 'Export bibliographic records'
7. Verify that exported records contain the 999$9 field
8. Repeat steps 3-7 with other export formats, and with authority
   records

Good patch, succeeded.

Signed-off-by: Devinim <kohadevinim@devinim.com.tr>
Comment 4 Jonathan Druart 2020-03-25 14:45:09 UTC
Julian, tests are failing for me.

    #   Failed test 'Export XML: marc modification template should have added 999$9'
    #   at t/db_dependent/Exporter/Record.t line 483.
    #          got: undef
    #     expected: 'Foobar'
    # Looks like you failed 1 test of 2.

#   Failed test 'export xml with marc modification template'
#   at t/db_dependent/Exporter/Record.t line 484.

    #   Failed test 'Export ISO2709: marc modification template should have added 999$9'
    #   at t/db_dependent/Exporter/Record.t line 508.
    #          got: undef
    #     expected: 'Foobar'
    # Looks like you failed 1 test of 2.

#   Failed test 'export iso2709 with marc modification template'
#   at t/db_dependent/Exporter/Record.t line 509.
# Looks like you failed 2 tests of 9.

Certainly coming from bug 21708.
Comment 5 Julian Maurice 2020-10-13 08:12:20 UTC
Created attachment 111519 [details] [review]
Bug 24679: Allow to apply a MARC modification template in export tool

Test plan:
1. Create a MARC modification template with a simple action (for
   instance, "Add new field 999$9 with value foobar")
2. Go to Tools » Export data
3. Select some records (for instance, from biblionumber 1 to 10)
4. In options, select the MARC modification template you just created
5. Select File format 'MARC'
6. Click on 'Export bibliographic records'
7. Verify that exported records contain the 999$9 field
8. Repeat steps 3-7 with other export formats, and with authority
   records

Signed-off-by: Devinim <kohadevinim@devinim.com.tr>
Comment 6 Julian Maurice 2020-10-13 08:12:25 UTC
Created attachment 111520 [details] [review]
Bug 24679: Return the correct id in AddModificationTemplate

'name' is not unique, and should not be used to retrieve the last
inserted id
Comment 7 Julian Maurice 2020-10-13 08:12:29 UTC
Created attachment 111521 [details] [review]
Bug 24679: Fix exporter tests

Use 998 instead of 999 which is used in MARC21 to store biblionumber
Comment 8 Katrin Fischer 2020-10-15 21:23:07 UTC
I am sorry, Julian, but tests are failing:

#   Failed test 'No tests run for subtest "export csv with marc modification template"'
#   at t/db_dependent/Exporter/Record.t line 441.
DBI last_insert_id: invalid number of arguments: got handle + 3, expected handle + between 4 and 5
Usage: $h->last_insert_id($catalog, $schema, $table_name, $field_name [, \%attr ]) at /home/vagrant/kohaclone/C4/MarcModificationTemplates.pm line 110.
# Looks like your test exited with 255 just after 7.
t/db_dependent/Exporter/Record.t .. Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 3/9 subtests 

Test Summary Report
-------------------
t/db_dependent/Exporter/Record.t (Wstat: 65280 Tests: 7 Failed: 1)
  Failed test:  7
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 9 tests but ran 7.
Comment 9 Fridolin Somers 2022-10-19 20:08:14 UTC
Up
Comment 10 David Cook 2023-01-08 23:07:36 UTC
This could help to add more feature parity with other library systems...
Comment 11 Fridolin Somers 2025-07-01 14:15:04 UTC
up ;)
Comment 12 Aleisha Amohia 2025-09-10 23:53:11 UTC
Created attachment 186343 [details] [review]
Bug 24679: Allow to apply a MARC modification template in export tool

Test plan:
1. Create a MARC modification template with a simple action (for
   instance, "Add new field 999$9 with value foobar")
2. Go to Tools » Export data
3. Select some records (for instance, from biblionumber 1 to 10)
4. In options, select the MARC modification template you just created
5. Select File format 'MARC'
6. Click on 'Export bibliographic records'
7. Verify that exported records contain the 999$9 field
8. Repeat steps 3-7 with other export formats, and with authority
   records

Signed-off-by: Devinim <kohadevinim@devinim.com.tr>
Comment 13 Aleisha Amohia 2025-09-10 23:53:14 UTC
Created attachment 186344 [details] [review]
Bug 24679: Return the correct id in AddModificationTemplate

'name' is not unique, and should not be used to retrieve the last
inserted id
Comment 14 Aleisha Amohia 2025-09-10 23:53:17 UTC
Created attachment 186345 [details] [review]
Bug 24679: Fix exporter tests

Tidy, and use 998 instead of 999 which is used in MARC21 to store biblionumber
Comment 15 Katrin Fischer 2026-01-26 18:09:01 UTC
The only real problem is the failing QA checks for tidiness. So I will sign-off and fail QA for that after. Please reset back to sign-off if it's only the formatting changes :)


1) QA checks: Patches require tidying (blocker)

testing 3 commit(s) (applied to 84227f7 'db8 Bug 41682: DBRev 25.12.00.014')

Processing files
[RUNNING] C4/MarcModificationTemplates.pm
[RUNNING] C4/Record.pm
[RUNNING] Koha/Exporter/Record.pm
[RUNNING] koha-tmpl/intranet-tmpl/prog/en/modules/tools/export.tt
[RUNNING] t/db_dependent/Exporter/Record.t
[RUNNING] tools/export.pl
[FAIL] C4/MarcModificationTemplates.pm
[PASS] t/db_dependent/Exporter/Record.t
[PASS] koha-tmpl/intranet-tmpl/prog/en/modules/tools/export.tt
[FAIL] C4/Record.pm
[FAIL] Koha/Exporter/Record.pm
[FAIL] tools/export.pl

Result:
[FAIL] C4/MarcModificationTemplates.pm
   FAIL	  tidiness
		File is not tidy, please run `perl misc/devel/tidy.pl C4/MarcModificationTemplates.pm`
[FAIL] C4/Record.pm
   FAIL	  tidiness
		File is not tidy, please run `perl misc/devel/tidy.pl C4/Record.pm`
[FAIL] Koha/Exporter/Record.pm
   FAIL	  tidiness
		File is not tidy, please run `perl misc/devel/tidy.pl Koha/Exporter/Record.pm`
[PASS] koha-tmpl/intranet-tmpl/prog/en/modules/tools/export.tt
[PASS] t/db_dependent/Exporter/Record.t
[FAIL] tools/export.pl
   FAIL	  tidiness
		File is not tidy, please run `perl misc/devel/tidy.pl tools/export.pl`

Processing additional checks

	* Signed-off-by email 'Signed-off-by: Devinim <kohadevinim@devinim.com.tr>' seems invalid


	* Was this bug sponsored by anyone?

Running tests (1)
	* Proving /kohadevbox/koha/t/db_dependent/Exporter/Record.t PASS


2) Code review  (OK)

a) CSV?

This updates marc2csv etc. but the test plan includes no info about exporting a CSV format. 

As the option only appears after a CSV profile is added, I feel it should be inlcuded: 
- Add a CSV profile of type "Export records"
- For example: Title=245|Test=999
- Export records using the CSV profile, verify MARC template is applied correctly


3) Testing

Tested with authority and bibliographic records, .mrc, .xml and .csv. It worked well!
Comment 16 Katrin Fischer 2026-01-26 18:09:51 UTC
Created attachment 192051 [details] [review]
Bug 24679: Allow to apply a MARC modification template in export tool

Test plan:
1. Create a MARC modification template with a simple action (for
   instance, "Add new field 999$9 with value foobar")
2. Go to Tools » Export data
3. Select some records (for instance, from biblionumber 1 to 10)
4. In options, select the MARC modification template you just created
5. Select File format 'MARC'
6. Click on 'Export bibliographic records'
7. Verify that exported records contain the 999$9 field
8. Repeat steps 3-7 with other export formats, and with authority
   records

Signed-off-by: Devinim <kohadevinim@devinim.com.tr>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 17 Katrin Fischer 2026-01-26 18:09:53 UTC
Created attachment 192052 [details] [review]
Bug 24679: Return the correct id in AddModificationTemplate

'name' is not unique, and should not be used to retrieve the last
inserted id

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 18 Katrin Fischer 2026-01-26 18:09:55 UTC
Created attachment 192053 [details] [review]
Bug 24679: Fix exporter tests

Tidy, and use 998 instead of 999 which is used in MARC21 to store biblionumber

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 19 Julian Maurice 2026-01-27 09:53:40 UTC
Created attachment 192091 [details] [review]
Bug 24679: Tidy

Patch from commit c4c5179