---- Reported by frederic@tamil.fr 2009-05-11 11:28:39 ---- In UNIMARC author facets aren't properly build. Personal Name: * 700$a contains author surname * 700$b contains author forenames Facets should be constructed by those two subfields. But just 700$a is used. ---- Additional Comments From frederic@tamil.fr 2010-05-14 12:54:53 ---- It implies modifying C4::Koha::getFacets and C4::Search. So it should rather be reported to 3.4 and C4::Search rewrite. --- Bug imported by chris@bigballofwax.co.nz 2010-05-21 01:07 UTC --- This bug was previously known as _bug_ 3216 at http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=3216 Actual time not defined. Setting to 0.0
Having just looked at the GetFacets code, I'm quite confident this is still an issue. Bumping the version number.
Created attachment 6509 [details] [review] Proposed patch To be tested on MARC21 and UNIMARC installations. Following patches welcomed to use subfields for other facets than UNIMARC 70x. On MARC21, nothing must change. To see a facet other subfields, tweak C4::Koha getFacets(). For example, to have 650 field other subfields, replace: { link_value => 'su-to', label_value => 'Topics', tags => ['650'], subfield => 'a', }, with: { link_value => 'su-to', label_value => 'Topics', tags => ['650'], subfield => 'abvxyz', },
This looks like it would work for UNIMARC 700 pretty well... but not so much for MARC21 650. The separator is hardcoded to ", "; for MARC21 650, it should be '--' usually. Also, if we removed the specific subfield hash key in getFacets, and attached the subfields directly to the fields, we'd be able to handle Item Type facets and any other condition where the faceted information is in different subfields of different fields: Examples: UNIMARC 700 { link_value => 'au', label_value => 'Authors', tags => [ '700ab', '701ab', '702ab', ], subfield_sep => ', ', } MARC21 650 { link_value => 'su-to', label_value => 'Topics', tags => ['650abvxyz'], subfield_sep => '--', } # this still won't solve the issue of ordering the subfields Item Types (MARC21) { link_value => 'itype', label_value => 'Item Types', tags => ['942c, 952y'], subfield_sep => '', },
(In reply to comment #3) > This looks like it would work for UNIMARC 700 pretty well... but not > so much for MARC21 650. The separator is hardcoded to ", "; for > MARC21 650, it should be '--' usually. You're correct. > Examples: > MARC21 650 > { > link_value => 'su-to', > label_value => 'Topics', > tags => ['650abvxyz'], > subfield_sep => '--', > } # this still won't solve the issue of ordering the subfields As it is coded, subfields are displayed as they are found in the field, which seems good for me. Isn't it? I will take a look and will implement your improved data structure. Be prepare to test...
Thanks, Frédéric! You're right, if the headings are coordinated in MARC21 650 as produced by Net::Z3950::Record->render(), then your code would preserve that. If not, well, nothing we can really do about it, but much better this than the way it is now.
Created attachment 6535 [details] [review] Proposed patch Better parameterizing as suggested... Following patches are required to handle better MARC21 subfields and choose other subfields to deal with UNIMARC format.
From a purely technical point of view, this works wonderfully. I was able to apply it, and searches continue to work just fine for MARC21. Unfortunately, as soon as I modify the 650 to be 650az, I run into a problem. The facet displays properly, but if I try to use it, I get no results. This is because, for now, Zebra does not allow multiple subfields to be concatenated and thrown into a single index. Searching for "su-to: Cookery -- Pennsylvania" will return nothing, even if that heading is plentiful in the catalog. Currently, the only way to get to that heading would be "su-to: Cookery AND su-to: Pennsylvania", or to use the linked authority record in subfield 9 I believe that this will mean that UNIMARC 700ab searching will fail; the author name will not be coordinated in the index. Unfortunately, I have no UNIMARC data to test locally, and the two demo sites in UNIMARC (BibLibre and Tamil) are not returning usable results. Can a UNIMARC library test this, and verify if it works? I don't think it will, but if it does, then there is something golden in the UNIMARC biblios/record.abs that I must steal for MARC21!
My initial testing of this on a UNIMARC system is very encouraging. I'm not sure how this is working; I work primarily with MARC21 configurations. If anyone UNIMARC user can sign off on this patch as working functionally, I'm confident that it can be passed through QA (code looks good, and enables some neat new functionality)
Created attachment 6716 [details] [review] ICU/CHR tokenization Word search with multi-part facets works properly only with Zebra ICU tokenization. This patch add a new question to Koha command line installer: Zebra has two methods to perform records tokenization and characters normalization: CHR and ICU. ICU is recommended for catalogs containing non-Latin characters. (chr, icu) [chr] How to test: - perl ./Makefile.PL - Try each possible value for new parameter - Take a look at zebradb/etc/default.idx file. Depending of the parameter you get this line: icuchain words-icu.xml or this one: charmap word-phrase-utf.chr
(In reply to comment #7) > Unfortunately, as soon as I modify the 650 to be 650az, I run into a problem. > The facet displays properly, but if I try to use it, I get no results. This is > because, for now, Zebra does not allow multiple subfields to be concatenated > and thrown into a single index. Searching for "su-to: Cookery -- Pennsylvania" > will return nothing, even if that heading is plentiful in the catalog. > Currently, the only way to get to that heading would be "su-to: Cookery AND > su-to: Pennsylvania", or to use the linked authority record in subfield 9 It works with Zebra ICU tokenization. The 2nd patch allow to install a Zebra default.idx file working with word and phrase ICU indexes.
Hi Frederic, both patches still apply. As this requires command line access, it can not be tested on Paul's sandbox system. Would it be possible for you to provide UNIMARC data for testing? Or I could try and sign off on this for MARC21 only.
Ok, trying to understand the discussion on this bug: so if someone wasn't using icu, but had multiple subfields defined in the facets - search will not work correctly?
(In reply to comment #12) > Ok, trying to understand the discussion on this bug: so if someone wasn't using > icu, but had multiple subfields defined in the facets - search will not work > correctly? Good question. Couldn't we dissociate both patch? We could add a new enhancement bug for the patch that add ICU support to the CLI installer. ICU works for sure current facets. So it wouldn't hurt anything to include it.
Testing on a UNIMARC sandbox: searching kw,wrdl: rue and au,wrdl: alain serres from advanced search yields 28 results. searching just kw,wrdl: rue yields 192 results. But clicking the "Serres, Alain" facet gives only 1 result! The query is constructing as limit=au:serres, alain, which could be the problem. I'm not sure if this is because the ICU requirements are incompatible with the sandbox set up, or if the patch is failing to work as expected.
Created attachment 8157 [details] [review] Bug 3216: UNIMARC author facets Add 700$b to UNIMARC author facets. Other facets subfields could be added now. For example, other subjects subfields. Following patches are required to handle better MARC21 subfields and choose other subfields to deal with UNIMARC format. Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com> Tested under both MARC21 and UNIMARC. Does not cause any regressions with MARC21, and offers the possibility for better faceting there in the future. Works as advertised with UNIMARC.
Makes faceting code much, much more flexible. Subfields from one field can now be mixed with different subfields from another (previously impossible). This allows, for example, a unified Item Type facet that pulls from both 942$c and 952$y. Marking, very happily, as Passed QA.
I like this patch very much ;-) However = wouldn't it be better to add 710a, 711a and 712a to authors ? patch pushed
Thanks for pushing this patch and the ICU one. > However = wouldn't it be better to add 710a, 711a and 712a to authors ? Yes. I purposely haven't introduced facets building modification, except 700$b subfield (and 606$x). It's possible to easily improve facets (and also in MARC21). We may have another bug entry for that. UNIMARC: For 710, 711, and 712, with subfields abcdef, another solution is to create for them a separate Corporate author facet. Another improvement is for 6xx tags, that have important subfields. For example 606ajxyz. Librarians should decide.
This bug will be included in the Koha 3.6.5 release.