On sites using DOM, when you delete a bib record, it doesn't go away immediately. When the fast indexer runs, the edit appeared in the search results, but the deleted record did not go away. It can take 3 or 4 days but does eventually disappear. This also causes problems when trying to import records because it's matching things that they've already deleted that really should be gone.
Could you add the exact cronjob invocation to this bug? I tried deleting bibs in a tiny test database that's using DOM indexing, both with and without the patch for bug 9496, and the records were deleted from the Zebra index when I ran rebuild_zebra.pl -b -z or rebuild_zebra.pl -b -x -z.
Galen, What we typically use on all sites is 'rebuild_zebra.pl -x -b -z'. - Larry
I am seeing the same issue. Example 1: delete all items run ./rebuild_zebra -b -z rerun search biblio is found with no items delete record run ./rebuild_zebra -b -z rerun search biblio with no items still shows up Example 2: delete all items and delete record run ./rebuild_zebra -b -z rerun search biblio with items will show up As a note: if "adelete" is replace with "delete" in you get "[warn] cannot delete record above (seems new)" for both cases above. The only way to correct the search results is to run ./rebuild_zebra -b -r It appears that the biblio record cannot be found by zebraidx to be removed.
I hope this might help: This record has been deleted. It still shows up on searches and when you click on it, you get "The record you requested does not exist (81828)." I found this using yaz-client (not easy). Z> show Sent presentRequest (20+1). Records: 1 [biblios]Record type: USmarc 01187nam a2200313Ia 4500 001 ocm56473895 003 OSt 005 20111031143426.0 008 040909s2004 nyua j 000 0 eng d 020 $a 0439455243 $c $3.50 040 $a OCO $c OCO $d STD $d OSt $d OhAtLVL 049 $a STDN 100 1 $a McCann, Jesse Leon. $9 1001550701 245 10 $a Scooby-doo and the werewolf / $c by Jesse Leon McCann. 246 3 $a Scooby-doo & the werewolf 260 $a New York : $b Scholastic, $c c2004. 300 $a 24 p. : $b ill. ; $c 22 cm. 440 00 $a Scooby-Doo! $9 3000254601 500 $a At head of title: Cartoon Network. 521 $a Level M (red) ; $b Scholastic Leveled Reading. 526 $a Accelerated reader ; $c 2.7 ; $d 0.5 ; 650 0 $a Scooby-Doo (Fictitious character) $v Juvenile fiction $9 2001577542 650 0 $a Leveled reader $9 100098602612 650 0 $a Accelerated Reader $9 100098597294 710 2 $a Cartoon Network (Television network) $9 1001407241 910 $a 267343 942 $c GE $k Scooby- Doo $2 ddc 999 $c 81828 $d 81828 952 $w 2007-02-25 $p 33529002654449 $v 3.50 $r 2012-12-01 $4 0 $0 0 $6 SCOOBY__DOO $9 95084 $b MAIN $m 12 $1 0 $o Scooby- Doo $d 2004-09-14 $7 0 $c JE $y GE $s 2012-11-10 $l 124 $a MAIN
What version of Koha are you using, Jorge?
Our production Koha is 3.10 but I reproduced the problem and took index data from a test server on master (3.11.00.014).
I can confirm (3.8.5) that rebuild_zebra with the incremental option -z, the biblio deletion does not occur; a full rebuild -b (with or without -x) is fully functional but is a *necessity*. This bug is [seriously?] compounded by overwriting a biblio with a Z39.50 import. This creates a second record in Zebra (not MySQL), identical biblionumber, callnumber, intemnumber: see <http://opac.navalmarinearchive.com/cgi-bin/koha/opac-search.pl?idx=callnum&q=GRLK-GREE-10> If a staff cataloguer is unaware of this bug and deletes the apparently incorrect (old) record, this deletes the good MySQL record. I therefore suggest upgrading this bug to "critical."
For those who are seeing this bug, could you paste the results of running the command zebraidx -V
(In reply to comment #8) > For those who are seeing this bug, could you paste the results of running > the command > > zebraidx -V Zebra 2.0.44 (C) 1994-2010, Index Data ApS Zebra is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. SHA1 ID: 419ad759807269fdfa379799a051ed3a551c6541 Using ICU
A comparison of the Zebra config of a database experiencing this problem shows that it lacked the patch for bug 8665. In particular, the Zebra record ID must be taken (for MARC21) from the 999$c, but prior to the patch for 8665, it was taken from the 001. This also explains why I couldn't reproduce the problem myself -- I was using master. Try updating biblio-koha-indexdefs.xml and biblio-zebra-indexdefs.xml, then do a full reindex of the bibs. After that is done, I suspect that rebuild_zebra.pl -z will start removing deleted bibs from the Zebra index.
For those on 3.8.x, the fix for 8665 was first released in 3.8.7.
We've got two libraries that were previously experiencing this problem on 3.8.5. We have upgraded them to 3.8.8, and both are now reporting that this problem looks to be resolved based on their testing so far. Thanks, Galen! Melia
3.8.10 tarball on Ubuntu 12.04.2 LTS, db imported from 3.8.5; using an incremental CRON '$KOHAPATH/bin/migration_tools/rebuild_zebra.pl -a -b -z -x' (with or without the '-x') does not remove the deleted record; a complete rebuild_zebra.pl -b -r -v -x is required.
(In reply to comment #13) > 3.8.10 tarball on Ubuntu 12.04.2 LTS, db imported from 3.8.5; using an > incremental CRON '$KOHAPATH/bin/migration_tools/rebuild_zebra.pl -a -b -z > -x' (with or without the '-x') does not remove the deleted record; a > complete rebuild_zebra.pl -b -r -v -x is required. Illustrated example of problem: Starting point: type='BK', title='The Hudson through the years', author='Adams, Arthur G.' Before intervention, we had *three* biblios, 1 for publisher='Fordham...', two for publisher='Lind...', each biblio with a single, valid 'item.' We did the following: - Transfer (copy barcode and "attach") a 'Lind' item from one biblio to the other and delete the "empty" biblio. - Overwrite the 'Lind' biblio now containing two items with a Z39.50 biblio from LoC (DLC.) After this, we had *four* [relevant] biblios in OPAC, see <http://navalmarinearchive.com/ims/opac_hudson_before_full_rebuild.jpg>. #1 as expected, untouched, #2 gave a 404 error (the deleted biblio), #3 was the *old* biblio with both items, #4 was the desired result (Z39.50 biblio with both items.) The "incremental" user=koha CRON ($KOHAPATH/bin/migration_tools/rebuild_zebra.pl -a -b -z) ran every minute overnight and did not correct this. Errors are redirected >> /tmp/cron_koha.log 2>&1 and no errors were reported. Running a ./bin/migration_tools/rebuild_zebra.pl -b -r -v -x this morning instantly corrected OPAC, see <http://navalmarinearchive.com/ims/opac_hudson_after_full_rebuild.jpg>.
Reading through the comments, this seems to be resolved since 3.8.7. Please reopen if I am mistaken and you still see this problem in current versions.