Bug 28528 - bulkmarcimport delete option doesn't delete biblio_metadata
Summary: bulkmarcimport delete option doesn't delete biblio_metadata
Status: Pushed to main
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: Main
Hardware: All All
: P5 - low major
Assignee: Roman Dolny
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-06-08 11:08 UTC by Katrin Fischer
Modified: 2026-02-26 16:06 UTC (History)
4 users (show)

See Also:
GIT URL:
Initiative type: ---
Sponsorship status: ---
Comma delimited list of Sponsors:
Crowdfunding goal: 0
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
26.05.00
Circulation function:


Attachments
Bug 28528: bulkmarcimport delete option doesn't delete biblio_metadata (2.18 KB, patch)
2025-01-20 20:36 UTC, Roman Dolny
Details | Diff | Splinter Review
Bug 28528: bulkmarcimport delete option doesn't delete biblio_metadata (2.24 KB, patch)
2025-04-03 21:29 UTC, Janusz Kaczmarek
Details | Diff | Splinter Review
Bug 28528: bulkmarcimport delete option doesn't delete biblio_metadata (2.33 KB, patch)
2026-02-20 09:00 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 28528: (QA follow-up) Remove "Koha 3" reference (806 bytes, patch)
2026-02-20 09:00 UTC, Marcel de Rooy
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Katrin Fischer 2021-06-08 11:08:54 UTC
When using bulkmarcimport with the -d switch should not only delete biblio, biblioitems and items, but also biblio_metadata.

I haven't checked if a FK takes care of the data, but if it does, we should still also reset the AUTO_INCREMENT:

https://git.koha-community.org/Koha-community/Koha/src/branch/master/misc/migration_tools/bulkmarcimport.pl#L172
Comment 1 Roman Dolny 2025-01-20 20:36:53 UTC
Created attachment 176844 [details] [review]
Bug 28528: bulkmarcimport delete option doesn't delete biblio_metadata

When using bulkmarcimport with the -d switch should not only delete
biblio, biblioitems and items, but also biblio_metadata.

I haven't checked if a FK takes care of the data, but if it does,
we should still also reset the AUTO_INCREMENT:...

Test plan:
==========
1. Check (and notice) the number of records in the biblio (bCount) and
   biblio_metadata (bmCount) tables. Check the AUTO_INCREMENT value
   for these tables (bAI, bmAI).
2. Import bibliographic records with delete, turn off foreign key checks
   and non-existent filename:
   misc/migration_tools/bulkmarcimport.pl -b -d -fk -file=non-exist-file
==> "Could not open..." is OK, file doesn't exist.
3. Repeat p. 1 and compare with previously noticed:
   bCount => 0 (OK)
   bmCount => unchanged (NOT OK)
   bAI => 1 (OK)
   bmAI => unchanged (NOT OK)
4. Apply the patch.
5. Repeat p. 2-3:
   bCount => 0 (OK)
   bmCount => 0 (changed, OK)
   bAI => 1 (OK)
   bmAI => 1 (changed, OK)

Sponsored-by: Ignatianum University in Cracow
Comment 2 Janusz Kaczmarek 2025-04-03 21:29:24 UTC
Created attachment 180606 [details] [review]
Bug 28528: bulkmarcimport delete option doesn't delete biblio_metadata

When using bulkmarcimport with the -d switch should not only delete
biblio, biblioitems and items, but also biblio_metadata.

I haven't checked if a FK takes care of the data, but if it does,
we should still also reset the AUTO_INCREMENT:...

Test plan:
==========
1. Check (and notice) the number of records in the biblio (bCount) and
   biblio_metadata (bmCount) tables. Check the AUTO_INCREMENT value
   for these tables (bAI, bmAI).
2. Import bibliographic records with delete, turn off foreign key checks
   and non-existent filename:
   misc/migration_tools/bulkmarcimport.pl -b -d -fk -file=non-exist-file
==> "Could not open..." is OK, file doesn't exist.
3. Repeat p. 1 and compare with previously noticed:
   bCount => 0 (OK)
   bmCount => unchanged (NOT OK)
   bAI => 1 (OK)
   bmAI => unchanged (NOT OK)
4. Apply the patch.
5. Repeat p. 2-3:
   bCount => 0 (OK)
   bmCount => 0 (changed, OK)
   bAI => 1 (OK)
   bmAI => 1 (changed, OK)

Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
Comment 3 Janusz Kaczmarek 2025-04-03 21:32:18 UTC
NB, without this patch after performing bulkmarcimport.pl -d -fk old content of biblio_metadata remains in the table!
Comment 4 Marcel de Rooy 2025-04-11 08:48:20 UTC
(In reply to Janusz Kaczmarek from comment #3)
> NB, without this patch after performing bulkmarcimport.pl -d -fk old content
> of biblio_metadata remains in the table!

Yeah, but thats probably the idea behind passing -fk to block the foreign key's cascaded delete.
Comment 5 Marcel de Rooy 2025-04-11 08:49:22 UTC
(In reply to Katrin Fischer from comment #0)
> When using bulkmarcimport with the -d switch should not only delete biblio,
> biblioitems and items, but also biblio_metadata.
> 
> I haven't checked if a FK takes care of the data, but if it does, we should
> still also reset the AUTO_INCREMENT:

Yes, it has a FK with delete.
Comment 6 Marcel de Rooy 2025-04-11 08:50:35 UTC
Moving to In Discussion. If you pass -fk and we are still going to delete, this may be confusing?
Comment 7 Janusz Kaczmarek 2026-02-18 15:09:43 UTC
(In reply to Marcel de Rooy from comment #6)
> Moving to In Discussion. If you pass -fk and we are still going to delete,
> this may be confusing?

If someone calls the script with -fk, we assume that they are
aware of the consequences and know what they are doing. As far as I 
remember, -fk appeared in this script when we introduced foreign 
keys on a larger scale, and execution with just -d became impossible 
in some cases (e.g., existing issues). 

We have two problems here, both solved with this patch:

1. Calling with -d, without -fk (when there are no dependencies
blocking the deletion of records) causes the biblio_metadata table 
to be empty, but AUTO_INCREMENT in biblio_metadata remains set to 
the old value and increments differently than biblionumber (which is 
not a big technical problem, but will raise unnecessary questions or 
doubts). 

2. Calling with -d and -fk causes biblio_metadata to retain rows 
that are not related to the biblio and biblioitems tables, which 
probably makes no sense. This is also not a technical problem, but 
it is a problem nonetheless, because this data takes up space 
completely unnecessarily and creates inconsistency in the data.
Comment 8 Marcel de Rooy 2026-02-20 09:00:41 UTC
Created attachment 193504 [details] [review]
Bug 28528: bulkmarcimport delete option doesn't delete biblio_metadata

When using bulkmarcimport with the -d switch should not only delete
biblio, biblioitems and items, but also biblio_metadata.

I haven't checked if a FK takes care of the data, but if it does,
we should still also reset the AUTO_INCREMENT:...

Test plan:
==========
1. Check (and notice) the number of records in the biblio (bCount) and
   biblio_metadata (bmCount) tables. Check the AUTO_INCREMENT value
   for these tables (bAI, bmAI).
2. Import bibliographic records with delete, turn off foreign key checks
   and non-existent filename:
   misc/migration_tools/bulkmarcimport.pl -b -d -fk -file=non-exist-file
==> "Could not open..." is OK, file doesn't exist.
3. Repeat p. 1 and compare with previously noticed:
   bCount => 0 (OK)
   bmCount => unchanged (NOT OK)
   bAI => 1 (OK)
   bmAI => unchanged (NOT OK)
4. Apply the patch.
5. Repeat p. 2-3:
   bCount => 0 (OK)
   bmCount => 0 (changed, OK)
   bAI => 1 (OK)
   bmAI => 1 (changed, OK)

Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 9 Marcel de Rooy 2026-02-20 09:00:43 UTC
Created attachment 193505 [details] [review]
Bug 28528: (QA follow-up) Remove "Koha 3" reference

Thats quite old :)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 10 Lucas Gass (lukeg) 2026-02-26 16:06:08 UTC
Nice work everyone!

Pushed to main for 26.05