Bug 3216 - UNIMARC author facets
Summary: UNIMARC author facets
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: OPAC (show other bugs)
Version: 3.6
Hardware: All All
: P2 normal (vote)
Assignee: Frédéric Demians
QA Contact: Ian Walls
URL:
Keywords:
Depends on: 7698
Blocks: 2629 4255 8958
  Show dependency treegraph
 
Reported: 2009-05-11 11:28 UTC by Frédéric Demians
Modified: 2013-12-05 20:01 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Proposed patch (3.66 KB, patch)
2011-12-02 07:17 UTC, Frédéric Demians
Details | Diff | Splinter Review
Proposed patch (9.88 KB, patch)
2011-12-02 15:07 UTC, Frédéric Demians
Details | Diff | Splinter Review
ICU/CHR tokenization (5.66 KB, patch)
2011-12-12 08:51 UTC, Frédéric Demians
Details | Diff | Splinter Review
Bug 3216: UNIMARC author facets (10.23 KB, patch)
2012-03-11 16:49 UTC, Jared Camins-Esakov
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Cormack 2010-05-21 01:07:55 UTC


---- Reported by frederic@tamil.fr 2009-05-11 11:28:39 ----

In UNIMARC author facets aren't properly build. 

Personal Name:

  * 700$a contains author surname
  * 700$b contains author forenames
  
Facets should be constructed by those two subfields. But
just 700$a is used.



---- Additional Comments From frederic@tamil.fr 2010-05-14 12:54:53 ----

It implies modifying C4::Koha::getFacets and C4::Search. So it should rather be reported to 3.4 and C4::Search rewrite.



--- Bug imported by chris@bigballofwax.co.nz 2010-05-21 01:07 UTC  ---

This bug was previously known as _bug_ 3216 at http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=3216

Actual time not defined. Setting to 0.0

Comment 1 Ian Walls 2011-12-01 22:50:22 UTC
Having just looked at the GetFacets code, I'm quite confident this is still an issue.  Bumping the version number.
Comment 2 Frédéric Demians 2011-12-02 07:17:08 UTC Comment hidden (obsolete)
Comment 3 Ian Walls 2011-12-02 14:14:51 UTC
This looks like it would work for UNIMARC 700 pretty well... but not so much for MARC21 650.  The separator is hardcoded to ", "; for MARC21 650, it should be '--' usually.

Also, if we removed the specific subfield hash key in getFacets, and attached the subfields directly to the fields, we'd be able to handle Item Type facets and any other condition where the faceted information is in different subfields of different fields:

Examples:

UNIMARC 700
{
    link_value  => 'au',
    label_value => 'Authors',
    tags        => [ '700ab', '701ab', '702ab', ],
    subfield_sep    => ', ',
}

MARC21 650
{
    link_value  => 'su-to',
    label_value => 'Topics',
    tags        => ['650abvxyz'],
    subfield_sep    => '--',
}  # this still won't solve the issue of ordering the subfields

Item Types (MARC21)
{
    link_value  => 'itype',
    label_value => 'Item Types',
    tags        => ['942c, 952y'],
    subfield_sep    => '',
},
Comment 4 Frédéric Demians 2011-12-02 14:27:29 UTC
(In reply to comment #3)
> This looks like it would work for UNIMARC 700 pretty well... but not
> so much for MARC21 650.  The separator is hardcoded to ", "; for
> MARC21 650, it should be '--' usually.

You're correct.

> Examples:

> MARC21 650
> {
>     link_value  => 'su-to',
>     label_value => 'Topics',
>     tags        => ['650abvxyz'],
>     subfield_sep    => '--',
> }  # this still won't solve the issue of ordering the subfields

As it is coded, subfields are displayed as they are found in the field,
which seems good for me. Isn't it?

I will take a look and will implement your improved data structure. Be
prepare to test...
Comment 5 Ian Walls 2011-12-02 14:33:37 UTC
Thanks, Frédéric!  You're right, if the headings are coordinated in MARC21 650 as produced by Net::Z3950::Record->render(), then your code would preserve that.  If not, well, nothing we can really do about it, but much better this than the way it is now.
Comment 6 Frédéric Demians 2011-12-02 15:07:21 UTC Comment hidden (obsolete)
Comment 7 Ian Walls 2011-12-02 19:07:47 UTC
From a purely technical point of view, this works wonderfully.  I was able to apply it, and searches continue to work just fine for MARC21.

Unfortunately, as soon as I modify the 650 to be 650az, I run into a problem.  The facet displays properly, but if I try to use it, I get no results.  This is because, for now, Zebra does not allow multiple subfields to be concatenated and thrown into a single index.  Searching for "su-to: Cookery -- Pennsylvania" will return nothing, even if that heading is plentiful in the catalog.  Currently, the only way to get to that heading would be "su-to: Cookery AND su-to: Pennsylvania", or to use the linked authority record in subfield 9

I believe that this will mean that UNIMARC 700ab searching will fail; the author name will not be coordinated in the index.  Unfortunately, I have no UNIMARC data to test locally, and the two demo sites in UNIMARC (BibLibre and Tamil) are not returning usable results.

Can a UNIMARC library test this, and verify if it works?  I don't think it will, but if it does, then there is something golden in the UNIMARC biblios/record.abs that I must steal for MARC21!
Comment 8 Ian Walls 2011-12-07 20:48:38 UTC
My initial testing of this on a UNIMARC system is very encouraging.  I'm not sure how this is working; I work primarily with MARC21 configurations.

If anyone UNIMARC user can sign off on this patch as working functionally, I'm confident that it can be passed through QA (code looks good, and enables some neat new functionality)
Comment 9 Frédéric Demians 2011-12-12 08:51:02 UTC Comment hidden (obsolete)
Comment 10 Frédéric Demians 2011-12-12 08:53:28 UTC
(In reply to comment #7)
> Unfortunately, as soon as I modify the 650 to be 650az, I run into a problem. 
> The facet displays properly, but if I try to use it, I get no results.  This is
> because, for now, Zebra does not allow multiple subfields to be concatenated
> and thrown into a single index.  Searching for "su-to: Cookery -- Pennsylvania"
> will return nothing, even if that heading is plentiful in the catalog. 
> Currently, the only way to get to that heading would be "su-to: Cookery AND
> su-to: Pennsylvania", or to use the linked authority record in subfield 9

It works with Zebra ICU tokenization. The 2nd patch allow to install a
Zebra default.idx file working with word and phrase ICU indexes.
Comment 11 Katrin Fischer 2012-02-24 06:52:46 UTC
Hi Frederic, 
both patches still apply. As this requires command line access, it can not be tested on Paul's sandbox system. Would it be possible for you to provide UNIMARC data for testing? Or I could try and sign off on this for MARC21 only.
Comment 12 Katrin Fischer 2012-02-24 06:59:36 UTC
Ok, trying to understand the discussion on this bug: so if someone wasn't using icu, but had multiple subfields defined in the facets - search will not work correctly?
Comment 13 Frédéric Demians 2012-02-24 07:10:36 UTC
(In reply to comment #12)
> Ok, trying to understand the discussion on this bug: so if someone wasn't using
> icu, but had multiple subfields defined in the facets - search will not work
> correctly?

Good question.

Couldn't we dissociate both patch? We could add a new enhancement bug for the patch that add ICU support to the CLI installer. ICU works for sure current facets. So it wouldn't hurt anything to include it.
Comment 14 Ian Walls 2012-03-02 17:33:59 UTC
Testing on a UNIMARC sandbox:

searching kw,wrdl: rue and au,wrdl: alain serres from advanced search yields 28 results.

searching just kw,wrdl: rue yields 192 results.  But clicking the "Serres, Alain" facet gives only 1 result!  The query is constructing as limit=au:serres, alain, which could be the problem.

I'm not sure if this is because the ICU requirements are incompatible with the sandbox set up, or if the patch is failing to work as expected.
Comment 15 Jared Camins-Esakov 2012-03-11 16:49:36 UTC
Created attachment 8157 [details] [review]
Bug 3216: UNIMARC author facets

Add 700$b to UNIMARC author facets.

Other facets subfields could be added now. For example, other subjects
subfields.

Following patches are required to handle better MARC21 subfields and choose
other subfields to deal with UNIMARC format.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Tested under both MARC21 and UNIMARC. Does not cause any regressions with
MARC21, and offers the possibility for better faceting there in the future.
Works as advertised with UNIMARC.
Comment 16 Ian Walls 2012-03-12 10:25:17 UTC
Makes faceting code much, much more flexible.  Subfields from one field can now be mixed with different subfields from another (previously impossible).  This allows, for example, a unified Item Type facet that pulls from both 942$c and 952$y.  Marking, very happily, as Passed QA.
Comment 17 Paul Poulain 2012-03-13 16:04:39 UTC
I like this patch very much ;-)

However = wouldn't it be better to add 710a, 711a and 712a to authors ?

patch pushed
Comment 18 Frédéric Demians 2012-03-13 19:32:28 UTC
Thanks for pushing this patch and the ICU one.

> However = wouldn't it be better to add 710a, 711a and 712a to authors ?

Yes. I purposely haven't introduced facets building modification, except
700$b subfield (and 606$x). It's possible to easily improve facets (and
also in MARC21). We may have another bug entry for that.

UNIMARC: For 710, 711, and 712, with subfields abcdef, another solution
is to create for them a separate Corporate author facet. Another
improvement is for 6xx tags, that have important subfields. For example
606ajxyz.

Librarians should decide.
Comment 19 Jared Camins-Esakov 2012-05-09 16:55:08 UTC
This bug will be included in the Koha 3.6.5 release.