Bug 16921 - Authority Linking doesn't match to proper record
Summary: Authority Linking doesn't match to proper record
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: MARC Authority data support (show other bugs)
Version: 16.05
Hardware: All All
: P5 - low major (vote)
Assignee: Galen Charlton
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-07-14 05:16 UTC by Nicholas van Oudtshoorn
Modified: 2019-06-27 09:24 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
New Linker Module: BestMatch (9.53 KB, patch)
2016-07-14 05:16 UTC, Nicholas van Oudtshoorn
Details | Diff | Splinter Review
New version of BestMatch linker, that includes using the cache (9.88 KB, patch)
2016-07-14 08:08 UTC, Nicholas van Oudtshoorn
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nicholas van Oudtshoorn 2016-07-14 05:16:31 UTC Comment hidden (obsolete)
Comment 1 Nicholas van Oudtshoorn 2016-07-14 08:08:28 UTC
Created attachment 53396 [details] [review]
New version of BestMatch linker, that includes using the cache

A slight tweak to the BestMatch linker, to use the cache - hopefully giving a slight speed bump!
Comment 2 Jacek Ablewicz 2016-07-14 10:48:35 UTC
(In reply to Nicholas van Oudtshoorn from comment #0)

> Copied from an email to the devel list:
> Investigating, it seems like zebra isn't giving the right search results
> back.
> 
> I can confirm this in yaz-client using the query that is created by koha.
> Searching for "Bentley, Peter, Ed." returns 4 results, only two of which are
> close. Actually, the record I'm trying to match is "Bentley, Peter, (Ed.)" -
> the last option.
> 
> Interestingly enough, if I search for "Bentley,
> 3432531531532513985735981<anything at all here>" it will also show me 
> the same four results.So does searching for "Bentley 513513531513515151".But
> searching for "Bentleyr435435151551251" shows no results, as I would expect.

Hi,
attribute combo (phrase, no trucation, complete field) used in yaz-client test, from the 1st mail in koha-devel list back in June 23:

    @attr 1=Match-heading @attr 4=1 @attr 5=100 @attr 6=3 "Bentley, Peter, Ed."

looks fine to me, and in my test setup it returns a proper result (only one, exact authority record matched). Most likely, problem is not in the Koha authority matching code, but in Zebra itself (it's config, or maybe the particular version you are using is buggy). BTW, do you have ICU tokenizer enabled? If yes, there were some known problems with it (completeness and position attributes not working at all, or not working properly) especially in not up-to-date zebra versions (see Bug 13751 and Bug 16581).
Comment 3 Joy Nelson 2016-07-14 13:55:43 UTC
I also am not seeing any oddities in my search results of authorities, nor any mismatches in the linker script.  Normally the oddities with the linker script are due to setting in the system preferences.   I have heard of other sites with issues with the linker that appeared to be a result of their zebra config.  
-joy


> Hi,
> attribute combo (phrase, no trucation, complete field) used in yaz-client
> test, from the 1st mail in koha-devel list back in June 23:
> 
>     @attr 1=Match-heading @attr 4=1 @attr 5=100 @attr 6=3 "Bentley, Peter,
> Ed."
> 
> looks fine to me, and in my test setup it returns a proper result (only one,
> exact authority record matched). Most likely, problem is not in the Koha
> authority matching code, but in Zebra itself (it's config, or maybe the
> particular version you are using is buggy). BTW, do you have ICU tokenizer
> enabled? If yes, there were some known problems with it (completeness and
> position attributes not working at all, or not working properly) especially
> in not up-to-date zebra versions (see Bug 13751 and Bug 16581).
Comment 4 Nicholas van Oudtshoorn 2016-07-21 07:33:01 UTC
Thanks Joy and Jacek,

Hmmm - I was on 2.0.58 of zebra - updated now, and things seem a lot better.  ICU is enabled - will try with it disabled at some point. However, I'm still having issues. 
For instance, searching for "Young, Robert" returns two results. One of them has a date in $d, the other doesn't. 
Another example, there's a PERSO_NAME of value "T" - and it is there, but koha's returning 104 results (anyone with the intital T in their name, it looks like!).

A more interesting example, searching for the TOPIC_TERM "Theology, Doctrinal" tells me there are three results:
These are:
    Theology, Doctrinal
    Theology, Doctrinal-popular Works
    Theology, Doctrinal.
Not sure what that second one is doing being brought in...

Is this data not particularly nice, and in need of a tidy up? Yep! But I can't understand why the search isn't returning the right results... any ideas?
Comment 5 David Cook 2016-08-01 23:29:03 UTC
(In reply to Nicholas van Oudtshoorn from comment #4)
> Thanks Joy and Jacek,
> 
> Hmmm - I was on 2.0.58 of zebra - updated now, and things seem a lot better.

>     Theology, Doctrinal
>     Theology, Doctrinal-popular Works
>     Theology, Doctrinal.
> Not sure what that second one is doing being brought in...
> 

Nicholas, are you on 2.0.59? If so, that hyphen is what's causing the problem. See bug 16581.

I've been warning the community for a long time that the version of Zebra in Debian is bad when using ICU, but those warnings have mostly been ignored.
Comment 6 David Cook 2016-08-01 23:30:09 UTC
(In reply to Joy Nelson from comment #3)
> I also am not seeing any oddities in my search results of authorities, nor
> any mismatches in the linker script.  Normally the oddities with the linker
> script are due to setting in the system preferences.   I have heard of other
> sites with issues with the linker that appeared to be a result of their
> zebra config.  
> -joy
> 

If I recall correctly, I think Bywater has often shown a preference for using IndexData's repositories for Zebra? So you might have a more up-to-date version of Zebra than Nicholas, I think.

Of course, this comment and my last one are after skimming this bug report for approximately 2 seconds, so I might be misreading :).
Comment 7 Nicholas van Oudtshoorn 2016-08-25 05:44:01 UTC
Hi all! Hmmmm - problems disappeared upgrading to zebra 2.0.62. (Forgot about this bug, actually!)
The version in fedora was a bit older than that - I've updated that too! :-)
Closing this bug - thanks for all the help everyone!