If a title has enough item record information to push it's MARCXML record size past 1MB, any information beyond that will be ignored. The record will be indexed, but the 999$c, which contains the biblionumber and is by its numerical nature the last field in the record, could be lost, and thus the link in the search results will be broken (no biblionumber filled in, so 404 error) The solution, ideally, would be to fix Net::Z3950::ZOOM to allow for the specification of a larger maximum record size, but that's impractical at this time. Koha can be altered to put all the 952 information AFTER the 999, practically guaranteeing that the biblionumber will be in the first 1MB worth of the MARCXML. If a few items records are lost to the indexer, that's unfortunate, but bearable.
Created attachment 5662 [details] [review] Proposed Patch See commit message... lots of details there.
Example record (cannot attach due to current attachment size limits): http://bywatersolutions.com/wp-content/uploads/2011/09/bug6789-test.xml_.txt Had to rename the extension to make the website happy
Created attachment 5679 [details] [review] Signed-off patch
Ian's solution here works like a charm! Tested with a real-world known-broken instance of this, and it's now working.
Created attachment 5886 [details] [review] Bug 6789: biblios with many items can result in broken search results link This patch fixes an issue whereby biblios with many items (often > 500) would index, but not the biblionumber itself, resulting in search results with a) inaccurate item counts and b) no biblionumber to use in the link to the details page. This is due to Net::Z3950::ZOOM not providing a mechanism for specifying different connection attributes; the maximumRecordSize ZOOM connection attribute, if not specified, defaults to 1MB, which is less than the size of a MARC record with many, many 952 fields. Since it is unlikely we can fix Net::Z3950::ZOOM in a timely fashion, this patch aims to build a workaround on the Koha end. This patch changes EmbedItemsInMarcBiblio to use append_fields instead of insert_ordered_fields, so the 999$c will come before the item records. It's VERY unlikely we will encounter more than 1MB of biblio-level MARC content, as this would break the ISO-2709 standard by a large factor. To this end, it also moves the fix_biblio_ids portion of get_corrected_marc_record out of rebuild_zebra.pl, and makes it a part of GetMarcBiblio (right before EmbedItemsInMarcBiblio, so the 952s still come last). fix_biblio_ids is kept as a subroutine for the deletion portion of rebuild_zebra.pl, which still uses it. It also uses the subroutine parameter in GetMarcBiblio to do the EmbedItemsInMarcBiblio action, rather than having rebuild_zebra.pl perform it on the itemless record returned from GetMarcBiblio. Simpler and cleaner that way. To verify bug issue: 1. Find a biblio with over 700 items (or enough that the resulting MARCXML is greater than 1MB) 2. search for this biblio (in a search that would return multiple results, not just this title). You should get the title in the results list 3. attempt to click the link to this biblio's details page; the biblionumber should be blank, leading to a 404 To test solution: 1. Apply patch 2. modify the biblio slightly (click the 005 for example) and save OR manually add the biblio to zebraqueue for reindexing 3. after rebuild_zebra.pl -z -b -x runs, use the same search as above. The title should still appear. 4. click the link, and find yourself on the biblio detail page as desired Signed-off-by: D Ruth Bavousett <ruth@bywatersolutions.com> Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
QA comment : 1- full test plan 2- change made in the core of the code but only a few lines, no side-effect I can see, except a light loss of performance caused by the _koha_marc_update_bib_ids call in GetMarcBiblio 3- MARC-independant, so works in UNIMARC as well (note in UNIMARC, the biblionumber is in 001, so the insert_ordered was not a problem) 4- Moving fix_biblio_ids into GetMarcBiblio is a good idea, and should/could be done for authorities as well : move the fix_authority_id into GetAuthority sub as well marked passed QA as the last comment is small compared to the problem this patch fixes. I suggest keeping the bug open until a fix is provided for my point 4
Paul, Thanks for the QA! I was very uncomfortable pushing this as a ByWater-only venture, so having an independent analysis from another community member (and one who can speak to UNIMARC support at that) eases my mind muchly.
Pushed please test.
This appears to be fixed. Closing.