In MARC 21 authority records, three bytes (008/14-16) indicate what the heading can be used for (main/added entry, subject entry, series entry). Koha should be able to respect the record's intended use in authority search during cataloging, provided that the authority file is well formed. Also, we should be able to decide between more strict but correct behavior and current lax behavior.
I think so too :) I can't remember whether I actually looked at whether you could do it with LinkerOptions rather than yet another pref, though. *** This bug has been marked as a duplicate of bug 33296 ***
This is not a duplicate to Bug 33296. Automated linker will be the next step. This is for cataloging authority finder, which is, technically, not connected with the linker.
Created attachment 174863 [details] [review] Bug 38494: Add 'ConsiderHeadingUse' system preference Add new system preference 'ConsiderHeadingUse' to by able to consider the intended use of authority records (008/14-16) during cataloging and linking.
Created attachment 174864 [details] [review] Bug 38494: Koha should consider authority heading use in cataloging In MARC 21 authority records, three bytes (008/14-16) indicate what the heading can be used for (main/added entry, subject entry, series entry). Koha should be able to respect the record's intended use in authority search during cataloging, provided that the authority file is well formed. Also, we should be able to decide between more strict but correct behavior and current lax behavior. Test plan: ---------- 1. Apply the patch; updatedatabase ; restart_all 2. Set the ShowHeadingUse system preference to ‘Show’ 3. Go to the Cataloging editor and open the authority finder plugin for the 100 field. Search for Peter. With standard ktd test data you should get 9 results, some with ‘v’ before Main/Added Entry, some with ‘x’. These with ‘x’ are not to be used in this context, provided you use a well-formed authority file 4. Set ConsiderHeadingUse system preference to ‘Do’ 5. Repeat p. 3. You should get now only 4 Peters – those with ‘v’ before Main/Added Entry (008/14 = ‘a’) 6. You can repeat the test for subject headings (6XX) and series headings (sometimes you would have to modify the test data, e.g. there are few authority records with 008/16 = ‘a’ (series use) 7. This should work equally with Zebra and with Elasticsearch
Created attachment 174865 [details] [review] Bug 38494: Add 'ConsiderHeadingUse' system preference Add new system preference 'ConsiderHeadingUse' to be able to consider the intended use of authority records (008/14-16) during cataloging and linking. Sponsored-by: Ignatianum University in Cracow
Created attachment 174866 [details] [review] Bug 38494: Koha should consider authority heading use in cataloging In MARC 21 authority records, three bytes (008/14-16) indicate what the heading can be used for (main/added entry, subject entry, series entry). Koha should be able to respect the record's intended use in authority search during cataloging, provided that the authority file is well formed. Also, we should be able to decide between more strict but correct behavior and current lax behavior. Test plan: ---------- 1. Apply the patch; updatedatabase ; restart_all 2. Set the ShowHeadingUse system preference to ‘Show’ 3. Go to the Cataloging editor and open the authority finder plugin for the 100 field. Search for Peter. With standard ktd test data you should get 9 results, some with ‘v’ before Main/Added Entry, some with ‘x’. These with ‘x’ are not to be used in this context, provided you use a well-formed authority file 4. Set ConsiderHeadingUse system preference to ‘Do’ 5. Repeat p. 3. You should get now only 4 Peters – those with ‘v’ before Main/Added Entry (008/14 = ‘a’) 6. You can repeat the test for subject headings (6XX) and series headings (sometimes you would have to modify the test data, e.g. there are few authority records with 008/16 = ‘a’ (series use) 7. This should work equally with Zebra and with Elasticsearch Sponsored-by: Ignatianum University in Cracow
I need to investigate a bit more how it represents in MARC for us, but I know that for GND records you can use the same records for both main/added and subject. So we don't have that differentiation in our authority file I believe.
(In reply to Katrin Fischer from comment #7) > I need to investigate a bit more how it represents in MARC for us, but I > know that for GND records you can use the same records for both main/added > and subject. So we don't have that differentiation in our authority file I > believe. Of course, it is totally correct. In such a case the authority record should have 008/14-15 = 'aa'. I am aware that there are libraries with authority files of low quality, self/automatically generated, not paying attention to 008. Therefor this feature should be optional, with default = 'do not'. For us it is essential because we are using more than one authority file for subject heading and only one for main/added headings. It happens that in different authority files there are names of equal form but only one file should be used for main/added entries. And this is controlled by 008/14-16. The story is in some sense analogous to the use of different thesauri (cf. Bug 30280). Official MARC 21 documentation regarding this topic: https://www.loc.gov/marc/authority/ad008.html
Created attachment 174897 [details] [review] Bug 38494: Koha should consider authority heading use in cataloging In MARC 21 authority records, three bytes (008/14-16) indicate what the heading can be used for (main/added entry, subject entry, series entry). Koha should be able to respect the record's intended use in authority search during cataloging, provided that the authority file is well formed. Also, we should be able to decide between more strict but correct behavior and current lax behavior. Test plan: ---------- 1. Apply the patch; updatedatabase ; restart_all 2. Set the ShowHeadingUse system preference to ‘Show’ 3. Go to the Cataloging editor and open the authority finder plugin for the 100 field. Search for Peter. With standard ktd test data you should get 9 results, some with ‘v’ before Main/Added Entry, some with ‘x’. These with ‘x’ are not to be used in this context, provided you use a well-formed authority file 4. Set ConsiderHeadingUse system preference to ‘Do’ 5. Repeat p. 3. You should get now only 4 Peters – those with ‘v’ before Main/Added Entry (008/14 = ‘a’) 6. You can repeat the test for subject headings (6XX) and series headings (sometimes you would have to modify the test data, e.g. there are few authority records with 008/16 = ‘a’ (series use) 7. This should work equally with Zebra and with Elasticsearch Sponsored-by: Ignatianum University in Cracow
Created attachment 174898 [details] [review] Bug 38494: Add 'ConsiderHeadingUse' system preference Add new system preference 'ConsiderHeadingUse' to be able to consider the intended use of authority records (008/14-16) during cataloging and linking. Sponsored-by: Ignatianum University in Cracow
Created attachment 174899 [details] [review] Bug 38494: Koha should consider authority heading use in cataloging In MARC 21 authority records, three bytes (008/14-16) indicate what the heading can be used for (main/added entry, subject entry, series entry). Koha should be able to respect the record's intended use in authority search during cataloging, provided that the authority file is well formed. Also, we should be able to decide between more strict but correct behavior and current lax behavior. Test plan: ---------- 1. Apply the patch; updatedatabase ; restart_all 2. Set the ShowHeadingUse system preference to ‘Show’ 3. Go to the Cataloging editor and open the authority finder plugin for the 100 field. Search for Peter. With standard ktd test data you should get 9 results, some with ‘v’ before Main/Added Entry, some with ‘x’. These with ‘x’ are not to be used in this context, provided you use a well-formed authority file 4. Set ConsiderHeadingUse system preference to ‘Do’ 5. Repeat p. 3. You should get now only 4 Peters – those with ‘v’ before Main/Added Entry (008/14 = ‘a’) 6. You can repeat the test for subject headings (6XX) and series headings (sometimes you would have to modify the test data, e.g. there are few authority records with 008/16 = ‘a’ (series use) 7. This should work equally with Zebra and with Elasticsearch Sponsored-by: Ignatianum University in Cracow
Created attachment 174914 [details] [review] Bug 38494: Add 'ConsiderHeadingUse' system preference Add new system preference 'ConsiderHeadingUse' to be able to consider the intended use of authority records (008/14-16) during cataloging and linking. Sponsored-by: Ignatianum University in Cracow Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Created attachment 174915 [details] [review] Bug 38494: Koha should consider authority heading use in cataloging In MARC 21 authority records, three bytes (008/14-16) indicate what the heading can be used for (main/added entry, subject entry, series entry). Koha should be able to respect the record's intended use in authority search during cataloging, provided that the authority file is well formed. Also, we should be able to decide between more strict but correct behavior and current lax behavior. Test plan: ---------- 1. Apply the patch; updatedatabase ; restart_all 2. Set the ShowHeadingUse system preference to ‘Show’ 3. Go to the Cataloging editor and open the authority finder plugin for the 100 field. Search for Peter. With standard ktd test data you should get 9 results, some with ‘v’ before Main/Added Entry, some with ‘x’. These with ‘x’ are not to be used in this context, provided you use a well-formed authority file 4. Set ConsiderHeadingUse system preference to ‘Do’ 5. Repeat p. 3. You should get now only 4 Peters – those with ‘v’ before Main/Added Entry (008/14 = ‘a’) 6. You can repeat the test for subject headings (6XX) and series headings (sometimes you would have to modify the test data, e.g. there are few authority records with 008/16 = ‘a’ (series use) 7. This should work equally with Zebra and with Elasticsearch Sponsored-by: Ignatianum University in Cracow Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Created attachment 177744 [details] [review] Bug 38494: Add 'ConsiderHeadingUse' system preference Add new system preference 'ConsiderHeadingUse' to be able to consider the intended use of authority records (008/14-16) during cataloging and linking. Sponsored-by: Ignatianum University in Cracow Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Created attachment 177745 [details] [review] Bug 38494: Koha should consider authority heading use in cataloging In MARC 21 authority records, three bytes (008/14-16) indicate what the heading can be used for (main/added entry, subject entry, series entry). Koha should be able to respect the record's intended use in authority search during cataloging, provided that the authority file is well formed. Also, we should be able to decide between more strict but correct behavior and current lax behavior. Test plan: ---------- 1. Apply the patch; updatedatabase ; restart_all 2. Set the ShowHeadingUse system preference to ‘Show’ 3. Go to the Cataloging editor and open the authority finder plugin for the 100 field. Search for Peter. With standard ktd test data you should get 9 results, some with ‘v’ before Main/Added Entry, some with ‘x’. These with ‘x’ are not to be used in this context, provided you use a well-formed authority file 4. Set ConsiderHeadingUse system preference to ‘Do’ 5. Repeat p. 3. You should get now only 4 Peters – those with ‘v’ before Main/Added Entry (008/14 = ‘a’) 6. You can repeat the test for subject headings (6XX) and series headings (sometimes you would have to modify the test data, e.g. there are few authority records with 008/16 = ‘a’ (series use) 7. This should work equally with Zebra and with Elasticsearch Sponsored-by: Ignatianum University in Cracow Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Rebased.
Could you please explain: + my $biblio_tag = substr( $index, 4, 3 ); + my $heading_use_search_field = + $biblio_tag =~ /^[127]/ ? 'Heading-use-main-or-added-entry' + : $biblio_tag =~ /^6/ ? 'Heading-use-subject-added-entry' + : $biblio_tag =~ /^[48]/ ? 'Heading-use-series-added-entry' How does index look? Couldnt find it so quickly. Maybe needing some comment here? + if ($heading_use_search_field) { + push @marclist, $heading_use_search_field; + push @and_or, 'and'; + push @excluding, ''; + push @operator, 'is'; + push @value, 'a'; + } What if it is marked No attempt to code? Thats something else as Not appropiate? Please clarify. Changing status for need of feedback. Thx.
008/14-16 is explained in https://www.loc.gov/marc/authority/ad008.html I consider that: Heading use-main or added entry = 100, 110, 111, 130, 240, 700, 710, 711, 730 Heading use-subject added entry = 600, 610, 611 ... Heading use-series added entry = 440 (if anybody is still using it), 800, 810, 811, 830 If the auth data is not precise, i.e. in 008/14-16 no attempt to code is used then the new syspref ConsiderHeadigUse should remain off (the default setting).
(In reply to Marcel de Rooy from comment #17) > How does index look? Couldnt find it so quickly. You can see 'index' parameter in the url of auth-finder.pl, e.g.: localhost:8081/cgi-bin/koha/authorities/auth_finder.pl?source=biblio&authtypecode=PERSO_NAME&index=tag_100_subfield_a_545156_359309&value_mainstr=&value_main= In this case: index=tag_100_subfield_a_545156_359309 so substr($index, 4, 3) gives you the biblio tag.
(In reply to Janusz Kaczmarek from comment #19) > (In reply to Marcel de Rooy from comment #17) > > How does index look? Couldnt find it so quickly. > > You can see 'index' parameter in the url of auth-finder.pl, e.g.: > > localhost:8081/cgi-bin/koha/authorities/auth_finder. > pl?source=biblio&authtypecode=PERSO_NAME&index=tag_100_subfield_a_545156_3593 > 09&value_mainstr=&value_main= > > In this case: index=tag_100_subfield_a_545156_359309 > > so substr($index, 4, 3) gives you the biblio tag. Thx. I should have known that :)
(In reply to Janusz Kaczmarek from comment #18) > 008/14-16 is explained in https://www.loc.gov/marc/authority/ad008.html > > I consider that: > Heading use-main or added entry = 100, 110, 111, 130, 240, 700, 710, 711, 730 > Heading use-subject added entry = 600, 610, 611 ... > Heading use-series added entry = 440 (if anybody is still using it), 800, > 810, 811, 830 > > If the auth data is not precise, i.e. in 008/14-16 no attempt to code is > used then the new syspref ConsiderHeadigUse should remain off (the default > setting). Thanks. Will add a follow-up to get this info in too.
Created attachment 179315 [details] [review] Bug 38494: Add 'ConsiderHeadingUse' system preference Add new system preference 'ConsiderHeadingUse' to be able to consider the intended use of authority records (008/14-16) during cataloging and linking. Sponsored-by: Ignatianum University in Cracow Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Created attachment 179316 [details] [review] Bug 38494: Koha should consider authority heading use in cataloging In MARC 21 authority records, three bytes (008/14-16) indicate what the heading can be used for (main/added entry, subject entry, series entry). Koha should be able to respect the record's intended use in authority search during cataloging, provided that the authority file is well formed. Also, we should be able to decide between more strict but correct behavior and current lax behavior. Test plan: ---------- 1. Apply the patch; updatedatabase ; restart_all 2. Set the ShowHeadingUse system preference to ‘Show’ 3. Go to the Cataloging editor and open the authority finder plugin for the 100 field. Search for Peter. With standard ktd test data you should get 9 results, some with ‘v’ before Main/Added Entry, some with ‘x’. These with ‘x’ are not to be used in this context, provided you use a well-formed authority file 4. Set ConsiderHeadingUse system preference to ‘Do’ 5. Repeat p. 3. You should get now only 4 Peters – those with ‘v’ before Main/Added Entry (008/14 = ‘a’) 6. You can repeat the test for subject headings (6XX) and series headings (sometimes you would have to modify the test data, e.g. there are few authority records with 008/16 = ‘a’ (series use) 7. This should work equally with Zebra and with Elasticsearch Sponsored-by: Ignatianum University in Cracow Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Created attachment 179317 [details] [review] Bug 38494: (QA follow-up) Add comment and extend pref desc See BZ comment18. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Some comments: 1) It appears this is a MARC21 only feature and it should be highlighted as such in the release notes and in the system preference description. 2) I think "no" should be 0 here? + default: no + choices: + 1: Do + 0: "Don't" Please review and follow-up.
(In reply to Katrin Fischer from comment #25) > Some comments: > > 1) It appears this is a MARC21 only feature and it should be highlighted as > such in the release notes and in the system preference description. > > 2) I think "no" should be 0 here? > > + default: no > + choices: > + 1: Do > + 0: "Don't" > > Please review and follow-up. Thank Katrin for reviewing. I am not sure, tbh. In authorities.pref I just followed a pattern from neighboring preferences. The default is expressed there with yes/no (I am not sure what impact and where these defaults may have -- in a fast code review I was not able to find any). As to the MARC21 note in preference description, I may add one, but again, the twin systempreference ConsiderHeadingUse which I looked at when creating this one does not mention MARC21. So in this case, shouldn't both rather be corrected in a separate follow up? What do you think?
(In reply to Janusz Kaczmarek from comment #26) > (In reply to Katrin Fischer from comment #25) > > Some comments: > > > > 1) It appears this is a MARC21 only feature and it should be highlighted as > > such in the release notes and in the system preference description. > > > > 2) I think "no" should be 0 here? > > > > + default: no > > + choices: > > + 1: Do > > + 0: "Don't" > > > > Please review and follow-up. > > Thank Katrin for reviewing. I am not sure, tbh. In authorities.pref I just > followed a pattern from neighboring preferences. The default is expressed > there with yes/no (I am not sure what impact and where these defaults may > have -- in a fast code review I was not able to find any). > > As to the MARC21 note in preference description, I may add one, but again, > the twin systempreference ConsiderHeadingUse which I looked at when creating > this one does not mention MARC21. So in this case, shouldn't both rather be > corrected in a separate follow up? What do you think? Hmmm, I believe maybe the default was supposed to be used with the editor, but since we also add an entry to systempreferences it just default to the database value? I suppose we can ignore that then. I think adding a note to both would be great as a quick follow-up here if you don't mind!
(In reply to Katrin Fischer from comment #27) > I think adding a note to both would be great as a quick follow-up here if > you don't mind! No problem. In this bug (ConsiderHeadingUse is a separate think), or (preferable) in a new bug?
I'd prefer here as a follow-up (text change, I'll QA)
Created attachment 179537 [details] [review] Bug 38494: (follow-up) Text amendments clarifying that it is about MARC21 records Refers to ConsiderHeadingUse and ShowHeadingUse systempreferences.
Created attachment 179543 [details] [review] Bug 38494: (follow-up) Text amendments clarifying that it is about MARC21 records Refers to ConsiderHeadingUse and ShowHeadingUse systempreferences.
Pushed for 25.05! Well done everyone, thank you!