Bug 14560 - Introduce a way of handling Rameau composed authorities, without breaking normal workflow
Summary: Introduce a way of handling Rameau composed authorities, without breaking nor...
Status: Patch doesn't apply
Alias: None
Product: Koha
Classification: Unclassified
Component: MARC Authority data support (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Matthias Meusburger
QA Contact: Testopia
URL:
Keywords:
: 8304 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-07-17 14:35 UTC by Matthias Meusburger
Modified: 2017-10-15 10:59 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Medium patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 14560: Rameau authorities (16.48 KB, patch)
2015-07-17 14:42 UTC, Matthias Meusburger
Details | Diff | Splinter Review
Bug 14560 [Follow-up]: Compatible with DOM indexing. (974 bytes, patch)
2015-08-04 13:06 UTC, Matthias Meusburger
Details | Diff | Splinter Review
Bug 14560 [Follow-up]: QA: Add debug warnings and get biblionumber with GetMarcFromKohaField (2.45 KB, patch)
2015-10-05 13:04 UTC, Matthias Meusburger
Details | Diff | Splinter Review
Bug 14560: Rameau authorities (16.29 KB, patch)
2017-01-12 12:18 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 14560 [Follow-up]: Compatible with DOM indexing. (968 bytes, patch)
2017-01-12 12:18 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 14560 [Follow-up]: QA: Add debug warnings and get biblionumber with GetMarcFromKohaField (2.45 KB, patch)
2017-01-12 12:18 UTC, Julian Maurice
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Meusburger 2015-07-17 14:35:35 UTC
The purpose of this patch is to give a clean support of the composed Rameau authorities.

This has been discussed a lot in Bug 8304, and led to a Failed QA (deserved, imho).

I've rewritten this from scratch, while keeping in mind to separate as much as possible the Rameau code from the legacy koha code.

What are Rameau authorities?
----------------------------

Rameau authorities are composed authorities that look like this:

606    _311954119
       _aGuerillas
       _311943195
       _yBolivie
       _311976947
       _z1945-1970

The purpose of this patch is to allow to add the koha internal links ($9) when merging the authority, while preserving the subfield order, which is meaningful.

If processed correctly (with this patch), the result after merge should be:

606    _311954119
       _926311
       _aGuerillas
       _311943195
       _926312
       _yBolivie
       _311976947
       _926313
       _z1945-1970
       _2rameau


How does it work?
-----------------

A new package has been introduced, C4/AuthoritiesRameau.pm, which is used only if the useRameau syspref is enabled.

Unit tests are here : t/db_dependent/AuthoritiesRameau.t

There are only 3 changes to the original merge code:

 1) Adds a new parameter to be able to specify if we want to merge to a specific biblio.
 2) Skip the legacy code if useRameau is enabled and there are multiple $9 in the field.
 3) use C4/AuthoritiesRameau.pm if useRameau is enabled.


Test plan
---------

It might be a little problematic for you to test this, because Rameau authorities are only used in France, so you may have to forge authorities if you don't have data with Rameau authorities. (or someone from France could test it ? :))

Anyway, make a call to the merge function like this:

merge($mergefrom,$MARCfrom,$mergeto,$MARCto, $biblionumber);

With $mergefrom and $mergeto being the same koha authid, and $MARCfrom and $MARCto being the result of GetAuthority($authid), and $biblionumber the biblio you want to merge the authority in.

I'm not sure there is a way to do this in the interface, if anyone knows...
Comment 1 Matthias Meusburger 2015-07-17 14:42:20 UTC
Created attachment 41072 [details] [review]
Bug 14560: Rameau authorities

The purpose of this patch is to give a clean support for merging composed
Rameau authorities.

Rameau authorities are composed authorities that look like this:

606    _311954119
       _aGuerillas
       _311943195
       _yBolivie
       _311976947
       _z1945-1970

If processed correctly, the result after merge should be:

606    _311954119
       _926311
       _aGuerillas
       _311943195
       _926312
       _yBolivie
       _311976947
       _926313
       _z1945-1970
       _2rameau

Test plan
---------

Make a call to the merge function like this:

merge($mergefrom,$MARCfrom,$mergeto,$MARCto, $biblionumber);

With $mergefrom and $mergeto being the same koha authid, $MARCfrom and $MARCto
being the result of GetAuthority($authid), and $biblionumber the biblio you
want to merge the authority in.
Comment 2 Katrin Fischer 2015-07-19 17:59:08 UTC
Hi Matthias, sorry, but I don't understand yet. 

You show the record before and after - but what is it merged with? I am not sure where the additional fields are from. Also, how will this be used? Your test plan is directed at another developer, but can this be triggered from within Koha? Is this for another development?
Comment 3 Matthias Meusburger 2015-07-22 10:31:39 UTC
Sorry, I will try to make this more understandable:

The before and after fields are the part of the biblio record in which a composed rameau authority has been merged to.

What we need to be done correctly during the merge is to add multiple $9, corresponding to the koha internal links to the authorities.

In this example:

This part of the 606 : 

_311954119
_aGuerillas

needs to be linked to the koha authority of authid 26311

This part of the 606 : 

_311943195
_yBolivie

needs to be linked to the koha authority of authid 26312

And this part of the 606 :

_311976947
_z1945-1970

needs to be linked to the koha authority of authid 26313


This is the reason for the extra $9 fields that are being added, leading to:

606    _311954119
       _926311
       _aGuerillas
       _311943195
       _926312
       _yBolivie
       _311976947
       _926313
       _z1945-1970
       _2rameau

The matching koha authority is found by matching the value of the external auth id in the biblio ($3) with the value of the external auth id in the authority (specified by the zebra index in the rameauBnfIdIndex syspref).


If this is still not clear enough, maybe a librarian could explain that better than I do.

And yes, currently this is used for an external tool ( see https://git.biblibre.com/biblibre/vendangeur ) we use to import records from the BnF Z3950 server ( see http://www.bnf.fr/en/professionals/bnf_data_services_tools/a.z3950_server_bnf_profile.html )

But this is the first and mandatory step for further developpments in koha, like supporting composed rameau authorities visual creation and edition in the admin interface.
Comment 4 Gaetan Boisson 2015-08-04 09:54:44 UTC
It is quite likely that composed authorities are a specificity of the French national library (BNF).

The idea is that, in a single authority controlled field of your bibliographic record, you will be linking to different authorities of your authority file.

In Matthias' example:

606    _311954119
       _aGuerillas
       _311943195
       _yBolivie
       _311976947
       _z1945-1970

The 606 field is for "a common noun or noun phrase used as a subject heading", but it allows for a $y Geographical Subdivision and a $z Chronological Subdivision.

The idea behind composed authorities is that the geographical subdivision will link to a different authority, which will be a Geographical Name, which would typically go in a 607 field as the main heading. So instead of having one authority for Guerillas in Bolivia, another one for Timor, yet another one for... etc. You have one Guerilla authority, and the subdivisions link to similarly unique authorities for geographical names.

For now this patch only aims at making this possible in Koha, and compatible with the tool we made to update one library's records with those from the BNF server's, available here: https://git.biblibre.com/biblibre/vendangeur. Managing this in the graphical interface would be the next step.
Comment 5 Matthias Meusburger 2015-08-04 13:06:50 UTC
Created attachment 41342 [details] [review]
Bug 14560 [Follow-up]: Compatible with DOM indexing.
Comment 6 claire.hernandez@biblibre.com 2015-09-29 12:30:18 UTC
*** Bug 8304 has been marked as a duplicate of this bug. ***
Comment 7 Matthias Meusburger 2015-10-05 13:04:39 UTC
Created attachment 43124 [details] [review]
Bug 14560 [Follow-up]: QA: Add debug warnings and get biblionumber with GetMarcFromKohaField
Comment 8 I'm just a bot 2016-03-08 18:32:16 UTC
.git/rebase-apply/patch:30: trailing whitespace.
    my ($counteditedbiblio,$countunmodifiedbiblio,$counterrors)=(0,0,0);        
.git/rebase-apply/patch:35: trailing whitespace.
    
.git/rebase-apply/patch:97: trailing whitespace.
    # Get All candidate Tags for the change 
.git/rebase-apply/patch:101: trailing whitespace.
    # May be used as a template for a bulkedit field  
.git/rebase-apply/patch:115: trailing whitespace.
                my $tag=$field->tag();          
warning: squelched 1 whitespace error
warning: 6 lines add whitespace errors.
.git/rebase-apply/patch:30: trailing whitespace.
    my ($counteditedbiblio,$countunmodifiedbiblio,$counterrors)=(0,0,0);        
.git/rebase-apply/patch:35: trailing whitespace.
    
.git/rebase-apply/patch:97: trailing whitespace.
    # Get All candidate Tags for the change 
.git/rebase-apply/patch:101: trailing whitespace.
    # May be used as a template for a bulkedit field  
.git/rebase-apply/patch:115: trailing whitespace.
                my $tag=$field->tag();          
error: patch failed: C4/AuthoritiesMarc.pm:1553
error: C4/AuthoritiesMarc.pm: patch does not apply
error: Did you hand edit your patch?
It does not apply to blobs recorded in its index. at ./getter.pl line 196.
Comment 9 Julian Maurice 2017-01-12 12:18:37 UTC
Created attachment 58871 [details] [review]
Bug 14560: Rameau authorities

Rebased on master
Comment 10 Julian Maurice 2017-01-12 12:18:46 UTC
Created attachment 58872 [details] [review]
Bug 14560 [Follow-up]: Compatible with DOM indexing.

Rebased on master
Comment 11 Julian Maurice 2017-01-12 12:18:56 UTC
Created attachment 58873 [details] [review]
Bug 14560 [Follow-up]: QA: Add debug warnings and get biblionumber with GetMarcFromKohaField

Rebased on master
Comment 12 Katrin Fischer 2017-10-15 10:59:46 UTC
I am very sorry Julian, it seems the code in AuthoritiesMarc.pm has moved quite a lot since the last rebase. The conflict doesn't look easy to resolve (to me at least).

Just a note to help with testing in the future:

It's possible to merge authorities in the GUI. Search your authorities, then from the action menu choose 'merge'. When you do that for the second record it will trigger the merge process.

Bug 14560 - Introduce a way of handling Rameau composed authorities, without breaking normal workflow

58871 - Bug 14560: Rameau authorities
58872 - Bug 14560 [Follow-up]: Compatible with DOM indexing.
58873 - Bug 14560 [Follow-up]: QA: Add debug warnings and get biblionumber with GetMarcFromKohaField

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 14560: Rameau authorities
Using index info to reconstruct a base tree...
M	C4/AuthoritiesMarc.pm
M	koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/authorities.pref
Falling back to patching base and 3-way merge...
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/authorities.pref
Auto-merging C4/AuthoritiesMarc.pm
CONFLICT (content): Merge conflict in C4/AuthoritiesMarc.pm
Failed to merge in the changes.
Patch failed at 0001 Bug 14560: Rameau authorities
The copy of the patch that failed is found in:
   /home/vagrant/kohaclone/.git/rebase-apply/patch
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-14560-Rameau-authorities-pIduys.patch