Bug 17788

Summary: (MARC21) $9 fields not indexed in authority-linked fields
Product: Koha Reporter: Tomás Cohen Arazi <tomascohen>
Component: MARC Bibliographic data supportAssignee: Tomás Cohen Arazi <tomascohen>
Status: CLOSED FIXED QA Contact: Marcel de Rooy <m.de.rooy>
Severity: normal    
Priority: P5 - low CC: fridolin.somers, hagud, julian.maurice, katrin.fischer, kyle, m.de.rooy, magnus, mtj
Version: master   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17547
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 17788: (MARC21) Add $9 fields to Koha-Auth-Number:w index
Bug 17788: (MARC21) Add $9 fields to Koha-Auth-Number:w index

Description Tomás Cohen Arazi 2016-12-17 12:24:38 UTC
If you check the default framework shipped with Koha, you will notice several fields are linked to authority records (even though they might be hidden in the editor):

MariaDB [koha_kohadev]> SELECT tagfield,tagsubfield,authtypecode FROM marc_subfield_structure WHERE authtypecode IS NOT NULL AND authtypecode<>'' AND frameworkcode='' GROUP BY tagfield,tagsubfield,authtypecode ;
+----------+-------------+--------------+
| tagfield | tagsubfield | authtypecode |
+----------+-------------+--------------+
| 100      | a           | PERSO_NAME   |
| 110      | a           | CORPO_NAME   |
| 111      | a           | MEETI_NAME   |
| 130      | a           | UNIF_TITLE   |
| 440      | a           | UNIF_TITLE   |
| 600      | a           | PERSO_NAME   |
| 610      | a           | CORPO_NAME   |
| 611      | a           | MEETI_NAME   |
| 630      | a           | UNIF_TITLE   |
| 648      | a           | CHRON_TERM   |
| 650      | a           | TOPIC_TERM   |
| 651      | a           | GEOGR_NAME   |
| 654      | a           | TOPIC_TERM   |
| 655      | a           | GENRE/FORM   |
| 656      | a           | TOPIC_TERM   |
| 657      | a           | TOPIC_TERM   |
| 658      | a           | TOPIC_TERM   |
| 662      | a           | GEOGR_NAME   |
| 690      | a           | TOPIC_TERM   |
| 691      | a           | GEOGR_NAME   |
| 696      | a           | PERSO_NAME   |
| 697      | a           | CORPO_NAME   |
| 698      | a           | MEETI_NAME   |
| 699      | a           | UNIF_TITLE   |
| 700      | a           | PERSO_NAME   |
| 710      | a           | CORPO_NAME   |
| 711      | a           | MEETI_NAME   |
| 730      | a           | UNIF_TITLE   |
| 796      | a           | PERSO_NAME   |
| 797      | a           | CORPO_NAME   |
| 798      | a           | MEETI_NAME   |
| 799      | a           | UNIF_TITLE   |
| 800      | a           | PERSO_NAME   |
| 810      | a           | CORPO_NAME   |
| 811      | a           | MEETI_NAME   |
| 830      | a           | UNIF_TITLE   |
| 896      | a           | PERSO_NAME   |
| 897      | a           | CORPO_NAME   |
| 898      | a           | MEETI_NAME   |
| 899      | a           | UNIF_TITLE   |
+----------+-------------+--------------+

But looking at etc/zebradb/marc_defs/marc21/biblio-koha-indexdefs.xml you will notice the $9 subfields is not correctly added to the Koha-Auth-Number:w index.

This has the consequence of authority usage count not being precise if those fields are used and linked.
Comment 1 Tomás Cohen Arazi 2016-12-17 12:32:45 UTC
Created attachment 58267 [details] [review]
Bug 17788: (MARC21) Add $9 fields to Koha-Auth-Number:w index

Looking at the default framework's fields that are linked to authority
records, there's a divergence with the Zebra index definitions.

This yields to authority usage count be incorrect for users searching
for authority records.

MariaDB [koha_kohadev]> SELECT tagfield,tagsubfield,authtypecode FROM
marc_subfield_structure WHERE authtypecode IS NOT NULL AND
authtypecode<>'' AND frameworkcode='' GROUP BY
tagfield,tagsubfield,authtypecode ;
+----------+-------------+--------------+
| tagfield | tagsubfield | authtypecode |
+----------+-------------+--------------+
| 100      | a           | PERSO_NAME   |
| 110      | a           | CORPO_NAME   |
| 111      | a           | MEETI_NAME   |
| 130      | a           | UNIF_TITLE   |
| 440      | a           | UNIF_TITLE   |
| 600      | a           | PERSO_NAME   |
| 610      | a           | CORPO_NAME   |
| 611      | a           | MEETI_NAME   |
| 630      | a           | UNIF_TITLE   |
| 648      | a           | CHRON_TERM   |
| 650      | a           | TOPIC_TERM   |
| 651      | a           | GEOGR_NAME   |
| 654      | a           | TOPIC_TERM   |
| 655      | a           | GENRE/FORM   |
| 656      | a           | TOPIC_TERM   |
| 657      | a           | TOPIC_TERM   |
| 658      | a           | TOPIC_TERM   |
| 662      | a           | GEOGR_NAME   |
| 690      | a           | TOPIC_TERM   |
| 691      | a           | GEOGR_NAME   |
| 696      | a           | PERSO_NAME   |
| 697      | a           | CORPO_NAME   |
| 698      | a           | MEETI_NAME   |
| 699      | a           | UNIF_TITLE   |
| 700      | a           | PERSO_NAME   |
| 710      | a           | CORPO_NAME   |
| 711      | a           | MEETI_NAME   |
| 730      | a           | UNIF_TITLE   |
| 796      | a           | PERSO_NAME   |
| 797      | a           | CORPO_NAME   |
| 798      | a           | MEETI_NAME   |
| 799      | a           | UNIF_TITLE   |
| 800      | a           | PERSO_NAME   |
| 810      | a           | CORPO_NAME   |
| 811      | a           | MEETI_NAME   |
| 830      | a           | UNIF_TITLE   |
| 896      | a           | PERSO_NAME   |
| 897      | a           | CORPO_NAME   |
| 898      | a           | MEETI_NAME   |
| 899      | a           | UNIF_TITLE   |
+----------+-------------+--------------+

This patch adds the missing ones to the authority number index as it is
done for the rest of the fields.

To test:
- Verify that
etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml
contains intries pointing the $9 subfield of all the fields in the
'tagfield' column above, to the Koha-Auth-Number:w index.
- Sign off :-D
Comment 2 Tomás Cohen Arazi 2016-12-17 12:34:24 UTC
I added known NORMARC and UNIMARC devs to CC, so they look at this issue and fill bugs for their MARC flavours if needed. This bug/patch covers MARC21.
Comment 3 Hugo Agud 2017-01-09 15:34:18 UTC
I have checked that all those fields are now indexed and this bugs it is signed off BUT under  my undestading this opens a new bug regarding the authorities

For example 650  SUBJECT ADDED ENTRY--TOPICAL TERM and 690 - LOCAL SUBJECT ADDED ENTRY--TOPICAL TERM (OCLC, RLIN)  POINTS TO the same authority base --> TOPIC_TERM

Under my understanding Koha should have new authority for cocver 690, 691, 692...If I am not wrong, If we don't add new authotiries, the official and local authorities will be the at the TOPIC_TERM and it is little weird

What do you think?
Comment 4 Katrin Fischer 2017-01-09 20:29:43 UTC
Linking the same authority type to multiple bibliographic fields should not cause any problem. As those are custom use fields, I think it's ok as it is. If someone needs a different framework for them, it can be added via the configuration.
Comment 5 Marcel de Rooy 2017-01-27 09:06:48 UTC
(In reply to Hugo Agud from comment #3)
> I have checked that all those fields are now indexed and this bugs it is
> signed off BUT under  my undestading this opens a new bug regarding the
> authorities
> 
> For example 650  SUBJECT ADDED ENTRY--TOPICAL TERM and 690 - LOCAL SUBJECT
> ADDED ENTRY--TOPICAL TERM (OCLC, RLIN)  POINTS TO the same authority base
> --> TOPIC_TERM
> 
> Under my understanding Koha should have new authority for cocver 690, 691,
> 692...If I am not wrong, If we don't add new authotiries, the official and
> local authorities will be the at the TOPIC_TERM and it is little weird
> 
> What do you think?

I do not have a strong opinion on that; you could open a new report and invite other people to give feedback too.
Comment 6 Marcel de Rooy 2017-01-27 09:11:23 UTC
Tomas: Why do we have 691 but not 791 (there is 751)
Comment 7 Marcel de Rooy 2017-01-27 09:13:22 UTC
Adding a signoff line too for Hugo Agud (check bug history)
Comment 8 Marcel de Rooy 2017-01-27 09:14:06 UTC
Created attachment 59604 [details] [review]
Bug 17788: (MARC21) Add $9 fields to Koha-Auth-Number:w index

Looking at the default framework's fields that are linked to authority
records, there's a divergence with the Zebra index definitions.

This yields to authority usage count be incorrect for users searching
for authority records.

MariaDB [koha_kohadev]> SELECT tagfield,tagsubfield,authtypecode FROM
marc_subfield_structure WHERE authtypecode IS NOT NULL AND
authtypecode<>'' AND frameworkcode='' GROUP BY
tagfield,tagsubfield,authtypecode ;
+----------+-------------+--------------+
| tagfield | tagsubfield | authtypecode |
+----------+-------------+--------------+
| 100      | a           | PERSO_NAME   |
| 110      | a           | CORPO_NAME   |
| 111      | a           | MEETI_NAME   |
| 130      | a           | UNIF_TITLE   |
| 440      | a           | UNIF_TITLE   |
| 600      | a           | PERSO_NAME   |
| 610      | a           | CORPO_NAME   |
| 611      | a           | MEETI_NAME   |
| 630      | a           | UNIF_TITLE   |
| 648      | a           | CHRON_TERM   |
| 650      | a           | TOPIC_TERM   |
| 651      | a           | GEOGR_NAME   |
| 654      | a           | TOPIC_TERM   |
| 655      | a           | GENRE/FORM   |
| 656      | a           | TOPIC_TERM   |
| 657      | a           | TOPIC_TERM   |
| 658      | a           | TOPIC_TERM   |
| 662      | a           | GEOGR_NAME   |
| 690      | a           | TOPIC_TERM   |
| 691      | a           | GEOGR_NAME   |
| 696      | a           | PERSO_NAME   |
| 697      | a           | CORPO_NAME   |
| 698      | a           | MEETI_NAME   |
| 699      | a           | UNIF_TITLE   |
| 700      | a           | PERSO_NAME   |
| 710      | a           | CORPO_NAME   |
| 711      | a           | MEETI_NAME   |
| 730      | a           | UNIF_TITLE   |
| 796      | a           | PERSO_NAME   |
| 797      | a           | CORPO_NAME   |
| 798      | a           | MEETI_NAME   |
| 799      | a           | UNIF_TITLE   |
| 800      | a           | PERSO_NAME   |
| 810      | a           | CORPO_NAME   |
| 811      | a           | MEETI_NAME   |
| 830      | a           | UNIF_TITLE   |
| 896      | a           | PERSO_NAME   |
| 897      | a           | CORPO_NAME   |
| 898      | a           | MEETI_NAME   |
| 899      | a           | UNIF_TITLE   |
+----------+-------------+--------------+

This patch adds the missing ones to the authority number index as it is
done for the rest of the fields.

To test:
- Verify that
etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml
contains intries pointing the $9 subfield of all the fields in the
'tagfield' column above, to the Koha-Auth-Number:w index.
- Sign off :-D

Signed-off-by: Hugo Agud <hagud@orex.es>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 9 Kyle M Hall 2017-02-14 13:58:43 UTC
Pushed to master for 17.05, thanks Tomas!
Comment 10 Katrin Fischer 2017-02-14 19:53:28 UTC
This patch has been pushed to 16.11.x and will be in 16.11.04.
Comment 11 Julian Maurice 2017-02-20 09:28:57 UTC
Pushed to 3.22.x for 3.22.17
Comment 12 Mason James 2017-02-23 20:20:49 UTC
Pushed to 16.05.x, for 16.05.10 release