Bugzilla – Attachment 146599 Details for
Bug 32916
[Bug 30280 follow-up] Problems in linking authorities to biblio fields (MARC 21)
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
[PATCH] Bug 32916: [Bug 30280 follow-up] Problems in linking authorities to biblio fields (MARC 21) - Zebra mangling
0001-Bug-32916-Bug-30280-follow-up-Problems-in-linking-Zebra.patch (text/plain), 15.68 KB, created by
Janusz Kaczmarek
on 2023-02-13 19:31:59 UTC
(
hide
)
Description:
[PATCH] Bug 32916: [Bug 30280 follow-up] Problems in linking authorities to biblio fields (MARC 21) - Zebra mangling
Filename:
MIME Type:
Creator:
Janusz Kaczmarek
Created:
2023-02-13 19:31:59 UTC
Size:
15.68 KB
patch
obsolete
>From 3240d4540cc51835fb05f478bcdf07435428bcfa Mon Sep 17 00:00:00 2001 >From: Janusz Kaczmarek <januszop@gmail.com> >Date: Mon, 13 Feb 2023 20:06:30 +0100 >Subject: [PATCH] Bug 32916: [Bug 30280 follow-up] Problems in linking > authorities to biblio fields (MARC 21) > >After applying the bug patch 30280: > >1. Koha does not link with Zebra. > >2. Koha does not properly link headings other than 6XX with ES. > >3. Koha does not link subject headings when 6XX indicator 2 = '4' or in >the case when in the auth record 008/11 = 'z', but 040 $f is not >defined, which is legal in terms of MARC 21 documentation >(https://www.loc.gov/marc/authority/ad008.html): "A MARC code for the >conventions used to formulate the heading may be contained in >subfield $f (Subject heading/thesaurus conventions) in field 040 >(Cataloging Source)." -- 'may', not 'should'. > >The new strict behaviour should be controlled (on/off) by a new >preference (e.g. LinkerStrictAuthInfo) -- since not every library is >using well formatted data. The semantics of this preference could be >expanded in the future to take into account also 008/14-16 of an auth >record for instance. > >Test plan: >========== > >1. Have a clean master (or 22.11.0[0-2]) installation > >2. Load provided data sample (bulkmarcimport.pl -d -a/b -file ...) > >3. Reindex with Zebra and ES > >4. Perform link_bibs_to_authorities.pl -v -t -l with Zebra, you >should get: > >Linked headings (from most frequent to least): >------------------------------------------------------- > >Unlinked headings (from most frequent to least): >------------------------------------------------------- >Feminism: 3 occurrences >Author 01: 1 occurrences >Author 01. Second work title: 1 occurrences >feminism: 1 occurrences >Person DBN: 1 occurrences >Series entry: 1 occurrences >Subject DBN without non-mandatory 040f: 1 occurrences >Subject lcsh: 1 occurrences >Subject with thesaurus not specified: 1 occurrences > >5. Perform link_bibs_to_authorities.pl -v -t -l with ES, you >should get: > >Linked headings (from most frequent to least): >------------------------------------------------------- >Feminism: 2 occurrences >feminism: 1 occurrences >Person DBN: 1 occurrences >Subject lcsh: 1 occurrences > >Unlinked headings (from most frequent to least): >------------------------------------------------------- >Author 01: 1 occurrences >Author 01. Second work title: 1 occurrences >Feminism: 1 occurrences >Series entry: 1 occurrences >Subject DBN without non-mandatory 040f: 1 occurrences >Subject with thesaurus not specified: 1 occurrences > >6. Apply the patch (pay attention to the location of the >authority-zebra-indexdefs.xsl file in you test environment). >Add a system preference LinkerStrictAuthInfo = 1. > >7. Full reindex with Zebra (koha-rebuild-zebra --full --force >-a -b) and ES (koha-elasticsearch --rebuild -r -d -a -b) > >8. Perform link_bibs_to_authorities.pl -v -t -l with >Zebra and ES, you should get in both cases: > >Linked headings (from most frequent to least): >------------------------------------------------------- >Feminism: 2 occurrences >Author 01: 1 occurrences >Author 01. Second work title: 1 occurrences >feminism: 1 occurrences >Person DBN: 1 occurrences >Series entry: 1 occurrences >Subject DBN without non-mandatory 040f: 1 occurrences >Subject lcsh: 1 occurrences >Subject with thesaurus not specified: 1 occurrences > >Unlinked headings (from most frequent to least): >------------------------------------------------------- >Feminism: 1 occurrences > >9. Perform link_bibs_to_authorities.pl (without -t) and >control the results in Koha -- all the heading fields >should be properly linked to the appropriate auth records. >--- > C4/AuthoritiesMarc.pm | 5 +- > C4/Heading.pm | 54 ++++++++--------- > C4/Heading/MARC21.pm | 26 +++++--- > .../Elasticsearch/QueryBuilder.pm | 1 - > .../searchengine/elasticsearch/mappings.yaml | 4 -- > .../authorities/authority-koha-indexdefs.xml | 9 ++- > .../authorities/authority-zebra-indexdefs.xsl | 60 ++++--------------- > 7 files changed, 62 insertions(+), 97 deletions(-) > >diff --git a/C4/AuthoritiesMarc.pm b/C4/AuthoritiesMarc.pm >index 0a2a736493..eaa0288f9a 100644 >--- a/C4/AuthoritiesMarc.pm >+++ b/C4/AuthoritiesMarc.pm >@@ -175,7 +175,10 @@ sub SearchAuthorities { > } > > my $operator = @$operator[$i]; >- if ( $operator and $operator eq 'is' ) { >+ if ( @$tags[$i] eq "thesaurus" ) { >+ $attr .= " \@attr 4=3 " >+ } >+ elsif ( $operator and $operator eq 'is' ) { > $attr .= " \@attr 4=1 \@attr 5=100 " > ; ##Phrase, No truncation,all of subfield field must match > } >diff --git a/C4/Heading.pm b/C4/Heading.pm >index 7541a56205..bf6d78ebe0 100644 >--- a/C4/Heading.pm >+++ b/C4/Heading.pm >@@ -194,8 +194,7 @@ sub _search { > my $self = shift; > my $index = shift || undef; > my $skipmetadata = shift || undef; >- my $ind2 = $self->{field}->{_ind2}; >- my $subject_heading_thesaurus = ''; >+ my $ind2 = $self->{field}->indicator('2'); > my @marclist; > my @and_or; > my @excluding = []; >@@ -209,36 +208,11 @@ sub _search { > push @value, $self->{'search_form'}; > } > >- if ($self->{'thesaurus'}) { >- if ($ind2 eq '0') { >- $subject_heading_thesaurus = 'a'; >- } elsif ($ind2 eq '1') { >- $subject_heading_thesaurus = 'b'; >- } elsif ($ind2 eq '2') { >- $subject_heading_thesaurus = 'c'; >- } elsif ($ind2 eq '3') { >- $subject_heading_thesaurus = 'd'; >- } elsif ($ind2 eq '4') { >- $subject_heading_thesaurus = 'n'; >- } elsif ($ind2 eq '5') { >- $subject_heading_thesaurus = 'k'; >- } elsif ($ind2 eq '6') { >- $subject_heading_thesaurus = 'v'; >- } else { >- $subject_heading_thesaurus = 'z'; >- } >+ if (C4::Context->preference("LinkerStrictAuthInfo") && $self->{'thesaurus'}) { > push @marclist, 'thesaurus'; > push @and_or, 'and'; > push @excluding, ''; > push @operator, 'is'; >- push @value, $subject_heading_thesaurus; >- } >- >- if ($ind2 eq '7') { >- push @marclist, 'thesaurus-conventions'; >- push @and_or, 'and'; >- push @excluding, ''; >- push @operator, 'is'; > push @value, $self->{'thesaurus'}; > } > >@@ -258,7 +232,29 @@ sub _search { > \@value, $self->{'auth_type'}, > 'AuthidAsc' > ); >- return $searcher->search_auth_compat( $search_query, 0, 20, $skipmetadata ); >+ my ( $authresults, $total ) = $searcher->search_auth_compat( $search_query, 0, 20, $skipmetadata ); >+ if (C4::Context->preference("LinkerStrictAuthInfo") && $self->{'thesaurus'} && ! $total && $ind2 eq '7') { >+ # For the case when auth 008/11 was 'z', but thesaurus was not declared in 040 $f >+ # (which is legal in MARC 21) we try to search again with 'other' (that stands >+ # for 008/11 = 'z') instead of biblio 6XX $7 (i.e. $self->{'thesaurus'}) >+ my $thesaurus_pos; >+ for my $i (0..$#marclist) { >+ if ($marclist[$i] eq 'thesaurus') { >+ $thesaurus_pos = $i; >+ last; >+ } >+ } >+ if ($thesaurus_pos) { >+ $value[$thesaurus_pos] = 'z'; >+ $search_query = $builder->build_authorities_query_compat( >+ \@marclist, \@and_or, \@excluding, \@operator, >+ \@value, $self->{'auth_type'}, >+ 'AuthidAsc' >+ ); >+ ( $authresults, $total ) = $searcher->search_auth_compat( $search_query, 0, 20, $skipmetadata ); >+ } >+ } >+ return ( $authresults, $total ); > } > > =head1 INTERNAL FUNCTIONS >diff --git a/C4/Heading/MARC21.pm b/C4/Heading/MARC21.pm >index 9ae5827541..302c3f1ed6 100644 >--- a/C4/Heading/MARC21.pm >+++ b/C4/Heading/MARC21.pm >@@ -339,7 +339,7 @@ sub parse_heading { > my $thesaurus = > $tag =~ m/6../ > ? _get_subject_thesaurus($field) >- : "lcsh"; # use 'lcsh' for names, UT, etc. >+ : undef; # defining thesaurus makes sence only for 6XX > my $search_heading = > _get_search_heading( $field, $field_info->{'subfields'} ); > my $display_heading = >@@ -361,29 +361,37 @@ sub _get_subject_thesaurus { > > my $thesaurus = "notdefined"; > if ( $ind2 eq '0' ) { >- $thesaurus = "lcsh"; >+ $thesaurus = "a"; > } > elsif ( $ind2 eq '1' ) { >- $thesaurus = "lcac"; >+ $thesaurus = "b"; > } > elsif ( $ind2 eq '2' ) { >- $thesaurus = "mesh"; >+ $thesaurus = "c"; > } > elsif ( $ind2 eq '3' ) { >- $thesaurus = "nal"; >+ $thesaurus = "d"; > } > elsif ( $ind2 eq '4' ) { >- $thesaurus = "notspecified"; >+ $thesaurus = "|"; > } > elsif ( $ind2 eq '5' ) { >- $thesaurus = "cash"; >+ $thesaurus = "k"; > } > elsif ( $ind2 eq '6' ) { >- $thesaurus = "rvm"; >+ $thesaurus = "v"; > } > elsif ( $ind2 eq '7' ) { > my $sf2 = $field->subfield('2'); >- $thesaurus = $sf2 if defined($sf2); >+ if (defined($sf2)) { >+ if ($sf2 eq 'sears') { >+ $thesaurus = 's'; >+ } elsif ($sf2 eq 'aat') { >+ $thesaurus = 'r'; >+ } else { >+ $thesaurus = $sf2; >+ } >+ } > } > > return $thesaurus; >diff --git a/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm b/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm >index 53042b4303..966f65fa4e 100644 >--- a/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm >+++ b/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm >@@ -559,7 +559,6 @@ our $koha_to_index_name = { > 'match-heading' => 'match-heading', > 'see-from' => 'match-heading-see-from', > thesaurus => 'subject-heading-thesaurus', >- 'thesaurus-conventions' => 'subject-heading-thesaurus-conventions', > any => '', > all => '' > }; >diff --git a/admin/searchengine/elasticsearch/mappings.yaml b/admin/searchengine/elasticsearch/mappings.yaml >index e892f9126c..378755aa45 100644 >--- a/admin/searchengine/elasticsearch/mappings.yaml >+++ b/admin/searchengine/elasticsearch/mappings.yaml >@@ -813,10 +813,6 @@ authorities: > marc_type: marc21 > sort: ~ > suggestible: '' >- type: '' >- Subject-heading-thesaurus-conventions: >- label: Subject-heading-thesaurus-conventions >- mappings: > - facet: '' > marc_field: 040f > marc_type: marc21 >diff --git a/etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml b/etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml >index 49f1c9f6f2..a7c0a1bb1c 100644 >--- a/etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml >+++ b/etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml >@@ -36,9 +36,9 @@ authority-zebra-indexdefs.xsl` > <kohaidx:index_control_field tag="008" offset="10" length="1"> > <kohaidx:target_index>Descriptive-cataloging-rules:w</kohaidx:target_index> > </kohaidx:index_control_field> >- <kohaidx:index_subject_thesaurus tag="008" offset="11" length="1" detail_tag="040" detail_subfields="f"> >- <kohaidx:target_index>Subject-heading-thesaurus:w</kohaidx:target_index> >- </kohaidx:index_subject_thesaurus> >+ <kohaidx:index_control_field tag="008" offset="11" length="1"> >+ <kohaidx:target_index>Subject-heading-thesaurus:0</kohaidx:target_index> >+ </kohaidx:index_control_field> > <kohaidx:index_control_field tag="008" offset="14" length="1"> > <kohaidx:target_index>Heading-use-main-or-added-entry:w</kohaidx:target_index> > </kohaidx:index_control_field> >@@ -66,6 +66,9 @@ authority-zebra-indexdefs.xsl` > <kohaidx:target_index>Record-source:w</kohaidx:target_index> > <kohaidx:target_index>Record-source:p</kohaidx:target_index> > </kohaidx:index_subfields> >+ <kohaidx:index_subfields tag="040" subfields="f"> >+ <kohaidx:target_index>Subject-heading-thesaurus:0</kohaidx:target_index> >+ </kohaidx:index_subfields> > > > <!-- Personal name headings --> >diff --git a/etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl b/etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl >index 0723caf2b4..8b0574b03a 100644 >--- a/etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl >+++ b/etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl >@@ -64,6 +64,9 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > <z:index name="Descriptive-cataloging-rules:w"> > <xslo:value-of select="substring(., 11, 1)"/> > </z:index> >+ <z:index name="Subject-heading-thesaurus:0"> >+ <xslo:value-of select="substring(., 12, 1)"/> >+ </z:index> > <z:index name="Heading-use-main-or-added-entry:w"> > <xslo:value-of select="substring(., 15, 1)"/> > </z:index> >@@ -100,6 +103,13 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > </z:index> > </xslo:if> > </xslo:for-each> >+ <xslo:for-each select="marc:subfield"> >+ <xslo:if test="contains('f', @code)"> >+ <z:index name="Subject-heading-thesaurus:0"> >+ <xslo:value-of select="."/> >+ </z:index> >+ </xslo:if> >+ </xslo:for-each> > </xslo:template> > <xslo:template mode="index_subfields" match="marc:datafield[@tag='100']"> > <xslo:for-each select="marc:subfield"> >@@ -2410,56 +2420,6 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > <xslo:variable name="general_subdivision_subfield">x</xslo:variable> > <xslo:variable name="chronological_subdivision_subfield">y</xslo:variable> > <xslo:variable name="geographic_subdivision_subfield">z</xslo:variable> >- <xslo:template mode="index_subject_thesaurus" match="marc:controlfield[@tag='008']"> >- <xslo:variable name="thesaurus_code1" select="substring(., 12, 1)"/> >- <xslo:variable name="full_thesaurus_code"> >- <xslo:choose> >- <xslo:when test="$thesaurus_code1 = 'a'"> >- <xslo:text>lcsh</xslo:text> >- </xslo:when> >- <xslo:when test="$thesaurus_code1 = 'b'"> >- <xslo:text>lcac</xslo:text> >- </xslo:when> >- <xslo:when test="$thesaurus_code1 = 'c'"> >- <xslo:text>mesh</xslo:text> >- </xslo:when> >- <xslo:when test="$thesaurus_code1 = 'd'"> >- <xslo:text>nal</xslo:text> >- </xslo:when> >- <xslo:when test="$thesaurus_code1 = 'k'"> >- <xslo:text>cash</xslo:text> >- </xslo:when> >- <xslo:when test="$thesaurus_code1 = 'n'"> >- <xslo:text>notapplicable</xslo:text> >- </xslo:when> >- <xslo:when test="$thesaurus_code1 = 'r'"> >- <xslo:text>aat</xslo:text> >- </xslo:when> >- <xslo:when test="$thesaurus_code1 = 's'"> >- <xslo:text>sears</xslo:text> >- </xslo:when> >- <xslo:when test="$thesaurus_code1 = 'v'"> >- <xslo:text>rvm</xslo:text> >- </xslo:when> >- <xslo:when test="$thesaurus_code1 = 'z'"> >- <xslo:choose> >- <xslo:when test="//marc:datafield[@tag='040']/marc:subfield[@code='f']"> >- <xslo:value-of select="//marc:datafield[@tag='040']/marc:subfield[@code='f']"/> >- </xslo:when> >- <xslo:otherwise> >- <xslo:text>notdefined</xslo:text> >- </xslo:otherwise> >- </xslo:choose> >- </xslo:when> >- <xslo:otherwise> >- <xslo:text>notdefined</xslo:text> >- </xslo:otherwise> >- </xslo:choose> >- </xslo:variable> >- <z:index name="Subject-heading-thesaurus:w"> >- <xslo:value-of select="$full_thesaurus_code"/> >- </z:index> >- </xslo:template> > <xslo:template mode="index_all" match="text()"> > <z:index name="Any:w Any:p"> > <xslo:value-of select="."/> >-- >2.30.2 >
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 32916
:
146523
|
146524
|
146598
| 146599