When using bulkmarcimport with the -d switch should not only delete biblio, biblioitems and items, but also biblio_metadata. I haven't checked if a FK takes care of the data, but if it does, we should still also reset the AUTO_INCREMENT: https://git.koha-community.org/Koha-community/Koha/src/branch/master/misc/migration_tools/bulkmarcimport.pl#L172
Created attachment 176844 [details] [review] Bug 28528: bulkmarcimport delete option doesn't delete biblio_metadata When using bulkmarcimport with the -d switch should not only delete biblio, biblioitems and items, but also biblio_metadata. I haven't checked if a FK takes care of the data, but if it does, we should still also reset the AUTO_INCREMENT:... Test plan: ========== 1. Check (and notice) the number of records in the biblio (bCount) and biblio_metadata (bmCount) tables. Check the AUTO_INCREMENT value for these tables (bAI, bmAI). 2. Import bibliographic records with delete, turn off foreign key checks and non-existent filename: misc/migration_tools/bulkmarcimport.pl -b -d -fk -file=non-exist-file ==> "Could not open..." is OK, file doesn't exist. 3. Repeat p. 1 and compare with previously noticed: bCount => 0 (OK) bmCount => unchanged (NOT OK) bAI => 1 (OK) bmAI => unchanged (NOT OK) 4. Apply the patch. 5. Repeat p. 2-3: bCount => 0 (OK) bmCount => 0 (changed, OK) bAI => 1 (OK) bmAI => 1 (changed, OK) Sponsored-by: Ignatianum University in Cracow
Created attachment 180606 [details] [review] Bug 28528: bulkmarcimport delete option doesn't delete biblio_metadata When using bulkmarcimport with the -d switch should not only delete biblio, biblioitems and items, but also biblio_metadata. I haven't checked if a FK takes care of the data, but if it does, we should still also reset the AUTO_INCREMENT:... Test plan: ========== 1. Check (and notice) the number of records in the biblio (bCount) and biblio_metadata (bmCount) tables. Check the AUTO_INCREMENT value for these tables (bAI, bmAI). 2. Import bibliographic records with delete, turn off foreign key checks and non-existent filename: misc/migration_tools/bulkmarcimport.pl -b -d -fk -file=non-exist-file ==> "Could not open..." is OK, file doesn't exist. 3. Repeat p. 1 and compare with previously noticed: bCount => 0 (OK) bmCount => unchanged (NOT OK) bAI => 1 (OK) bmAI => unchanged (NOT OK) 4. Apply the patch. 5. Repeat p. 2-3: bCount => 0 (OK) bmCount => 0 (changed, OK) bAI => 1 (OK) bmAI => 1 (changed, OK) Sponsored-by: Ignatianum University in Cracow Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
NB, without this patch after performing bulkmarcimport.pl -d -fk old content of biblio_metadata remains in the table!
(In reply to Janusz Kaczmarek from comment #3) > NB, without this patch after performing bulkmarcimport.pl -d -fk old content > of biblio_metadata remains in the table! Yeah, but thats probably the idea behind passing -fk to block the foreign key's cascaded delete.
(In reply to Katrin Fischer from comment #0) > When using bulkmarcimport with the -d switch should not only delete biblio, > biblioitems and items, but also biblio_metadata. > > I haven't checked if a FK takes care of the data, but if it does, we should > still also reset the AUTO_INCREMENT: Yes, it has a FK with delete.
Moving to In Discussion. If you pass -fk and we are still going to delete, this may be confusing?
(In reply to Marcel de Rooy from comment #6) > Moving to In Discussion. If you pass -fk and we are still going to delete, > this may be confusing? If someone calls the script with -fk, we assume that they are aware of the consequences and know what they are doing. As far as I remember, -fk appeared in this script when we introduced foreign keys on a larger scale, and execution with just -d became impossible in some cases (e.g., existing issues). We have two problems here, both solved with this patch: 1. Calling with -d, without -fk (when there are no dependencies blocking the deletion of records) causes the biblio_metadata table to be empty, but AUTO_INCREMENT in biblio_metadata remains set to the old value and increments differently than biblionumber (which is not a big technical problem, but will raise unnecessary questions or doubts). 2. Calling with -d and -fk causes biblio_metadata to retain rows that are not related to the biblio and biblioitems tables, which probably makes no sense. This is also not a technical problem, but it is a problem nonetheless, because this data takes up space completely unnecessarily and creates inconsistency in the data.
Created attachment 193504 [details] [review] Bug 28528: bulkmarcimport delete option doesn't delete biblio_metadata When using bulkmarcimport with the -d switch should not only delete biblio, biblioitems and items, but also biblio_metadata. I haven't checked if a FK takes care of the data, but if it does, we should still also reset the AUTO_INCREMENT:... Test plan: ========== 1. Check (and notice) the number of records in the biblio (bCount) and biblio_metadata (bmCount) tables. Check the AUTO_INCREMENT value for these tables (bAI, bmAI). 2. Import bibliographic records with delete, turn off foreign key checks and non-existent filename: misc/migration_tools/bulkmarcimport.pl -b -d -fk -file=non-exist-file ==> "Could not open..." is OK, file doesn't exist. 3. Repeat p. 1 and compare with previously noticed: bCount => 0 (OK) bmCount => unchanged (NOT OK) bAI => 1 (OK) bmAI => unchanged (NOT OK) 4. Apply the patch. 5. Repeat p. 2-3: bCount => 0 (OK) bmCount => 0 (changed, OK) bAI => 1 (OK) bmAI => 1 (changed, OK) Sponsored-by: Ignatianum University in Cracow Signed-off-by: Janusz Kaczmarek <januszop@gmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Created attachment 193505 [details] [review] Bug 28528: (QA follow-up) Remove "Koha 3" reference Thats quite old :) Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Nice work everyone! Pushed to main for 26.05