Bug 41508 - bulkmarcimport.pl: Add option to clean deleted* tables when -d is used
Summary: bulkmarcimport.pl: Add option to clean deleted* tables when -d is used
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-12-31 13:27 UTC by Katrin Fischer
Modified: 2026-01-07 10:01 UTC (History)
2 users (show)

See Also:
GIT URL:
Initiative type: ---
Sponsorship status: ---
Comma delimited list of Sponsors:
Crowdfunding goal: 0
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Katrin Fischer 2025-12-31 13:27:09 UTC
As bulkmarcimport is mostly used for doing the initial loading of a database and already allows to delete all bibliographic and authority records using the -d parameter, it would also make sense to offere an option to clean the deleted* tables in the same step as starting a new import.

Maybe this should be optional as bulkmarcimport also provides some matching capabilities that indicate it's not always used on an empty db. But if you delete "all" with -d it might still be a safe assumption? 

Having some warnings in place about the change might be good.
Comment 1 Alexander Wagner 2026-01-05 15:25:17 UTC
I wonder: if you delete in the db here, shouldn't this also happen to the search index (whatever one is used)? Just as I ran into some funny behaviour the other day that was cured by cleaning up the Zebras.
Comment 2 Katrin Fischer 2026-01-05 15:34:17 UTC
It's an interesting question.
We used to stop the koha-indexer and do a full index after, that will reset properly. If you want to use the koha-indexer, you can reset the indexes before importing, we use:

$ sudo koha-shell -l SITE
$ zebraidx -c /etc/koha/sites/SITE/zebra-biblios-dom.cfg init
$ zebraidx -c /etc/koha/sites/SITE/zebra-authorities-dom.cfg init

I am not sure if that could be built into the script, but it woudld certainly be for another bug. :)
Comment 3 Alexander Wagner 2026-01-05 16:02:29 UTC
> We used to stop the koha-indexer and do a full index after, that will reset properly.

This is what my installer does as well. Most likely we even discussed the steps back when I added this.

I was just wondering as your initial bug heads quite a bit towards "make it consistent" and it might not be too obvious that the index isn't cleared if you `bulkmarcimport -d`.
Comment 4 Katrin Fischer 2026-01-07 09:48:57 UTC
(In reply to Alexander Wagner from comment #3)
> > We used to stop the koha-indexer and do a full index after, that will reset properly.
> 
> This is what my installer does as well. Most likely we even discussed the
> steps back when I added this.
> 
> I was just wondering as your initial bug heads quite a bit towards "make it
> consistent" and it might not be too obvious that the index isn't cleared if
> you `bulkmarcimport -d`.

I was just trying to keep the changes required as small as possible, as I probably won't be able to work on this anytime soon. I think handling the indexes could be a separate bug as we also need to check Zebra and/or Elasticsearch. Maybe a visible hint/reminder when the script is run would help for a start?
Comment 5 Alexander Wagner 2026-01-07 10:01:21 UTC
> Maybe a visible hint/reminder when the script is run would help for a start?

Sounds good. It avoids that you assume some magic that doesn't happen.