Bug 13760

Summary: Authorities merge sometimes create duplicate fields in biblio
Product: Koha Reporter: Fridolin Somers <fridolin.somers>
Component: CatalogingAssignee: Fridolin Somers <fridolin.somers>
Status: CLOSED DUPLICATE QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: abl, jonathan.druart, katrin.fischer, m.de.rooy, nick
Version: master   
Hardware: All   
OS: All   
See Also: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13762
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11700
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14583
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 13760 - Authorities merge sometimes create duplicate fields in biblio
Bug 13760 - Authorities merge sometimes create duplicate fields in biblio
Bug 13760 - formatting code

Description Fridolin Somers 2015-02-25 15:17:02 UTC
When merging two authorities the biblio records linked to those authorities are updated. It is directly done if syspref "dontmerge" is on "Do".

Actually some cases create a duplicate field.
For example, a biblio record as : 
  600 $9 11 $a Son Goku
  600 $9 12 $a Vegeta
If you merge the authorities id=11 and id=12 into id=11 (creating Gogeta), you actually get :
  600 $9 11 $a Son Goku
  600 $9 11 $a Son Goku

Having two identical fields linked to the same authority is useless.

The same problem is even more frequent when merging two authorities of different type.
Comment 1 Fridolin Somers 2015-02-25 15:40:10 UTC Comment hidden (obsolete)
Comment 2 Marcel de Rooy 2015-03-02 14:17:31 UTC
(In reply to Fridolin SOMERS from comment #1)
> Note that this patch changes the minium of code in order to rebase it
> easyly. A followup will do the code formatting.

I do not think that this is the recommended approach.
If you put some part of the code in an if-statement, you need to indent.

Note that part of the same code still contains tabs too..
Comment 3 Fridolin Somers 2015-03-13 09:26:42 UTC
Created attachment 36846 [details] [review]
Bug 13760 - Authorities merge sometimes create duplicate fields in biblio

When merging two authorities the biblio records linked to those authorities are updated. It is directly done if syspref "dontmerge" is on "Do".

Actually some cases create a duplicate field.
For example, a biblio record as :
  600 $9 11 $a Son Goku
  600 $9 12 $a Vegeta
If you merge the authorities id=11 and id=12 into id=11 (creating Gogeta), you actually get :
  600 $9 11 $a Son Goku
  600 $9 11 $a Son Goku

Having two identical fields linked to the same authority is useless.

The same problem is even more frequent when merging two authorities of different type.

This patch corrects this behavior by searching already existing field with the link to the autority, using flag $field_to_exists. This flag is not used in case the merge is after a single authority modification.

Note that this patch changes the minium of code in order to rebase it easyly. A followup will do the code formatting.

Test plan :
- Set syspref dontmerge to 'Do'
- Create a biblio record
- Link a field to an authority (the field gets a $9). For example :
  600 $9 11 $a Son Goku
- Link a field to another authority (the field gets a $9). For example :
  600 $9 12 $a Vegeta
- Save the record
- Index Zebra queue
- Search for autority "Vegeta"
- Click on "Merge"
- Search for autority "Son Goku"
- Click on "Merge"
- Click on "Next"
- Click on "Merge"
- Go to biblio record
=> Without the patch : you see 2 fields linked to the same authority "Son Goku"
=> With the patch : you see 1 field linked to the authority "Son Goku"
- Test also with autoritites of different type.
- Check that modifying the autority "Son Goku" impacts the biblio record
Comment 4 Fridolin Somers 2015-03-13 09:26:54 UTC
Created attachment 36847 [details] [review]
Bug 13760 - formatting code
Comment 5 Fridolin Somers 2015-03-13 09:28:06 UTC
Here is the follow-up patch for code formatting with perltidy.
Comment 6 Jacek Ablewicz 2015-04-01 10:39:16 UTC
(In reply to Fridolin SOMERS from comment #0)

> Actually some cases create a duplicate field.
> For example, a biblio record as : 
>   600 $9 11 $a Son Goku
>   600 $9 12 $a Vegeta

> If you merge the authorities id=11 and id=12 into id=11 (creating Gogeta),
> you actually get :
>   600 $9 11 $a Son Goku
>   600 $9 11 $a Son Goku

This patch seems to work fine for relativelly simple cases. But what about some more complex ones, like e.g:

   700 $9 11 $a Son Goku $t Aaa Bbb
   700 $9 12 $a Vegeta $t Ddd Ccc
   700 $9 12 $a Vegeta $t Eee Fff

after merge, result will be:

   700 $9 11 $a Son Goku $t Aaa Bbb

instead of

   700 $9 11 $a Son Goku $t Aaa Bbb
   700 $9 11 $a Son Goku $t Ddd Ccc
   700 $9 11 $a Son Goku $t Eee Fff

> Having two identical fields linked to the same authority is useless.

I agree, but there is no check anywhere to ensure that the suspected "duplicates" are in fact identical? While having 2+ $9 links to the same authority record from one biblio record is probably not very common (depending on the material type, local cataloguing rules etc.), AFAIK this can happen occasionally and in such cases it's not always necessarily an error and/or a duplicate.
Comment 7 Fridolin Somers 2015-06-29 06:50:53 UTC
(In reply to Jacek Ablewicz from comment #6)
> (In reply to Fridolin SOMERS from comment #0)

Indeed, I agree.
thanks for your help
Comment 8 Marcel de Rooy 2017-01-14 12:09:57 UTC
(In reply to Jacek Ablewicz from comment #6)
>    700 $9 11 $a Son Goku $t Aaa Bbb
>    700 $9 12 $a Vegeta $t Ddd Ccc
>    700 $9 12 $a Vegeta $t Eee Fff

> While having 2+ $9 links to the same
> authority record from one biblio record is probably not very common
> (depending on the material type, local cataloguing rules etc.), AFAIK this
> can happen occasionally and in such cases it's not always necessarily an
> error and/or a duplicate.

Can anyone confirm that this case is a valid one with reference to cataloguing rules etc.?

This discussion comes up actually on several of the reports in this category. Suppose that we would have a 'strict' merge mode and a 'loose' merge mode, that either removes all extra subfields or leaves them untouched in the biblio record while merging, would that be sufficient? See also the AuthCleanBiblios pref of bug 5572; I would btw rather call it something like AuthorityMergeMode.
Comment 9 Marcel de Rooy 2017-01-14 12:45:52 UTC
I am closing this report in favor of more comprehensive solutions under 5572/11315/11700.
See omnibus bug 17908.
Thanks for your work.

*** This bug has been marked as a duplicate of bug 17908 ***
Comment 10 Katrin Fischer 2017-01-15 11:09:21 UTC
>    700 $9 11 $a Son Goku $t Aaa Bbb
>    700 $9 12 $a Vegeta $t Ddd Ccc
>    700 $9 12 $a Vegeta $t Eee Fff

It's a bit tricky to find a good explanation about it, but this seems related to how compilations are handled with RDA:
https://www.loc.gov/aba/rda/source/special_topics_compilations.ppt

So it's valid to have multiple links to the same authority in one bibliographic record, as long as the $t (title of work) is different. 

I have seen examples of that in German records recently too.