Bug 18017

Summary: [Unimarc] Importing biblios from Z39.50 servers duplicates related authorities
Product: Koha Reporter: Alex Arnaud <alex.arnaud>
Component: CatalogingAssignee: Alex Arnaud <alex.arnaud>
Status: Needs Signoff --- QA Contact: Testopia <testopia>
Severity: major    
Priority: P5 - low CC: fridolin.somers, jonathan.druart, laurence.rault, m.de.rooy, victor.grousset
Version: unspecified   
Hardware: All   
OS: All   
Whiteboard:
Change sponsored?: --- Patch complexity: ---
Bot Control: --- When did the bot last check this:
Who signed the patch off: Text to go in the release notes:
Attachments: Bug 18017 - add index_matching_heading to create indexes for matching an entire authority heading

Description Alex Arnaud 2017-01-31 09:35:14 UTC
This is a unimarc bug only:

When importing biblios from Z39.50, related authorities are wrong matched by the linker. This causes duplicates of authorities. The problem come from a bad marc definition in unimarc/authorities/authority-koha-indexdefs.xml which does not create a index for matching an entire heading.
Comment 1 Alex Arnaud 2017-01-31 09:38:52 UTC
Created attachment 59675 [details] [review]
Bug 18017 - add index_matching_heading to create indexes for matching an entire authority heading
Comment 2 Jonathan Druart 2017-02-16 13:21:21 UTC
A test plan to help testers?
Comment 3 Laurence Rault 2017-07-28 14:58:50 UTC
If I understand the patch, a test plan could be

Test plan

BiblioAddsAuthorities ON
AutoCreateAuthorities ON

1) Import Z3950 record and save it : authorities are created
2) Import the same record again, when saving say it is not a duplicate : 
Without the patch : new authorities (duplicates) are created for the same values
With the patch : new record should be linked with existing authorities



Tested on sandbox : 

BiblioAddsAuthorities ON
AutoCreateAuthorities ON

1) Import Z3950 record and save it : authorities are created
2) Import the same record again, when saving say it is not a duplicate : 
Without the patch : new authorities (duplicates) are created for the same values
With the patch : new authorities (duplicates) are created for the same values : KO
Comment 4 Alex Arnaud 2017-11-17 11:13:46 UTC
Full test plan is:

BiblioAddsAuthorities ON
AutoCreateAuthorities ON

1) replace /koha_conf_dir/etc/zebradb/marc_defs/unimarc/authorities/authority-koha-indexdefs.xml
   with /koha_clone/etc/zebradb/marc_defs/unimarc/authorities/authority-koha-indexdefs.xml
2) replace /koha_conf_dir/etc/zebradb/marc_defs/unimarc/authorities/authority-zebra-indexdefs.xsl
   with /koha_clone/etc/zebradb/marc_defs/unimarc/authorities/authority-zebra-indexdefs.xsl
3) reindex authorities,
4) Import Z3950 record and save it : authorities are created
5) Import the same record again, when saving say it is not a duplicate : 
Without the patch : new authorities (duplicates) are created for the same values
With the patch : new record should be linked with existing authorities
Comment 5 Victor Grousset/tuxayo 2017-11-24 14:46:47 UTC
After applying the patch, when reimporting, the authority of the main author is still duplicated. (different id on the "Main Author" link)

I even re-indexed authorities after applying the patch.
./src/misc/migration_tools/rebuild_zebra.pl -r -a -v
Comment 6 Victor Grousset/tuxayo 2018-05-11 07:05:55 UTC
Retried:

1) Set sysprefs
   BiblioAddsAuthorities ON
   AutoCreateAuthorities ON
2) Replace xsl files
     cp $KOHA_CLONE/etc/zebradb/marc_defs/unimarc/authorities/authority-koha-indexdefs.xml $KOHA_CONF_DIR/zebradb/marc_defs/unimarc/authorities/authority-koha-indexdefs.xml
     cp $KOHA_CLONE/etc/zebradb/marc_defs/unimarc/authorities/authority-zebra-indexdefs.xsl $KOHA_CONF_DIR/zebradb/marc_defs/unimarc/authorities/authority-zebra-indexdefs.xsl
3) reindex authorities
     ./misc/migration_tools/rebuild_zebra.pl -r -a -v
4) Import Z3950 record and save it : authorities are created
     go to staff:/cgi-bin/koha/cataloguing/addbooks.pl
5) Reimport the same record (when asked, say that it's not a duplicate)
6) The authority should have been duplicated (different url)
7) Apply this patch
8) Replace again the xsl files
9) reindex authorities
10) Reimport the same record
11) The authority should have not been duplicated
      But for me it was.