Bug 41508

Summary: bulkmarcimport.pl: Add option to clean deleted* tables when -d is used
Product: Koha Reporter: Katrin Fischer <katrin.fischer>
Component: Command-line UtilitiesAssignee: Bugs List <koha-bugs>
Status: NEW --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: alexander.wagner, robin
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41008
GIT URL: Initiative type: ---
Sponsorship status: --- Comma delimited list of Sponsors:
Crowdfunding goal: 0 Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Description Katrin Fischer 2025-12-31 13:27:09 UTC
As bulkmarcimport is mostly used for doing the initial loading of a database and already allows to delete all bibliographic and authority records using the -d parameter, it would also make sense to offere an option to clean the deleted* tables in the same step as starting a new import.

Maybe this should be optional as bulkmarcimport also provides some matching capabilities that indicate it's not always used on an empty db. But if you delete "all" with -d it might still be a safe assumption? 

Having some warnings in place about the change might be good.
Comment 1 Alexander Wagner 2026-01-05 15:25:17 UTC
I wonder: if you delete in the db here, shouldn't this also happen to the search index (whatever one is used)? Just as I ran into some funny behaviour the other day that was cured by cleaning up the Zebras.
Comment 2 Katrin Fischer 2026-01-05 15:34:17 UTC
It's an interesting question.
We used to stop the koha-indexer and do a full index after, that will reset properly. If you want to use the koha-indexer, you can reset the indexes before importing, we use:

$ sudo koha-shell -l SITE
$ zebraidx -c /etc/koha/sites/SITE/zebra-biblios-dom.cfg init
$ zebraidx -c /etc/koha/sites/SITE/zebra-authorities-dom.cfg init

I am not sure if that could be built into the script, but it woudld certainly be for another bug. :)
Comment 3 Alexander Wagner 2026-01-05 16:02:29 UTC
> We used to stop the koha-indexer and do a full index after, that will reset properly.

This is what my installer does as well. Most likely we even discussed the steps back when I added this.

I was just wondering as your initial bug heads quite a bit towards "make it consistent" and it might not be too obvious that the index isn't cleared if you `bulkmarcimport -d`.
Comment 4 Katrin Fischer 2026-01-07 09:48:57 UTC
(In reply to Alexander Wagner from comment #3)
> > We used to stop the koha-indexer and do a full index after, that will reset properly.
> 
> This is what my installer does as well. Most likely we even discussed the
> steps back when I added this.
> 
> I was just wondering as your initial bug heads quite a bit towards "make it
> consistent" and it might not be too obvious that the index isn't cleared if
> you `bulkmarcimport -d`.

I was just trying to keep the changes required as small as possible, as I probably won't be able to work on this anytime soon. I think handling the indexes could be a separate bug as we also need to check Zebra and/or Elasticsearch. Maybe a visible hint/reminder when the script is run would help for a start?
Comment 5 Alexander Wagner 2026-01-07 10:01:21 UTC
> Maybe a visible hint/reminder when the script is run would help for a start?

Sounds good. It avoids that you assume some magic that doesn't happen.