Bugzilla – Attachment 179007 Details for
Bug 38336
Confusion between 099 and 942 field in UNIMARC framework
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 38336: Fix confusion between 099 and 942 UNIMARC fields
Bug-38336-Fix-confusion-between-099-and-942-UNIMAR.patch (text/plain), 19.36 KB, created by
Mathieu Saby
on 2025-03-06 09:35:28 UTC
(
hide
)
Description:
Bug 38336: Fix confusion between 099 and 942 UNIMARC fields
Filename:
MIME Type:
Creator:
Mathieu Saby
Created:
2025-03-06 09:35:28 UTC
Size:
19.36 KB
patch
obsolete
>From 108ce1a4d9fdf1c1cb4fbd98400a47cb28a447c9 Mon Sep 17 00:00:00 2001 >From: Mathieu Saby <mathsabypro@gmail.com> >Date: Tue, 14 Jan 2025 18:02:54 +0000 >Subject: [PATCH] Bug 38336: Fix confusion between 099 and 942 UNIMARC fields > >This patch fixes 099 and 942 fields (and incidentally 200$b) >in the default and the FA UNIMARC frameworks and provides >the changes that follow logically : >- Zebra and Elasticsearch indexing >- removal of the unused TYPEDOC authorized values category >- UNIMARC facets in Koha.pm >- test in TestBuilder.pm > >These changes are minimal and are intended to correct only >the most serious problems. Subsequent work may bring improvements. > >Detail of changes : >- The 099 field is renamed "Local dates (Koha)" and should be >used in default configuration only for this storing creation >and modification dates of record. >- The 942 field is mandatory (because its $c subfield is) >and should be used for all other Koha local information, >like its MARC21 counterpart. >- The 200$b field is no more indexed (it is now obsolete in >cataloguing standards, and its MARC21 counterpart is not indexed). >- Like in MARC21, ccode is now defined only at item level and >itemtype/itype is now defined at item and biblio level. >- The default values for itemtype at biblio and item level are >the same, controlled by itype values (document type). >- The TYPEDOC authorized values category, which was used for >controlling the value if itemtype at record level, is removed. >- In Koha.pm UNIMARC facets are fixed to be a mirror of MARC21 >facets: a facet for itemtypes is created, and the facet for ccode >is now based only on items. >- A test on record-level itemtype in TestBuilder.pm is fixed >in order to replace 099 with 942 field. > >Test plan: > >Note that sample data in ktd and sandbox will need to be fixed once >this patch is pushed (there is no data in 942$c for example). >It may cause warnings if you edit existing record but it is not a bug. > >A. Check AV and framework changes >1/ In a UNIMARC k-t-d or sandbox, with Zebra search engine >2/ Apply the patch >3/ Check that there is no "TYPEDOC" authorized value category >4/ Check default and FA framework > - 099 should be labelled "Local dates (Koha)" and > contain 2 subfields:c and d > - 942 should be mandatory, labelled "ADDED ENTRY ELEMENTS (Koha)" > and contain 5 subfields : 0,2,6,c (mandatory),s >5/ Create a new record with default framework > -- add a 200$a field, a 100$a field (with the cataloging plugin) > -- in 942$c field, check that the values in dropdown list are > the one defines for 'item types' in administration (Books, > Computer Files, Continuing Resource, etc). > -- Choose the value "Computer field" and Save the record > -- Koha displays the item creation form. Check that the > preselected value in 995$r (item type) is "Computer field" >6/ Do the same operation for a new record with FA framework > >B. Check Zebra indexing changes >7 / Search "livre" in the catalog. 419 results should be shown >8 / Check that an "Item type" facet is visible and functional >9 / Edit the first record of the result list, change its 942$c >value and save >10 / Edit an item of that record and change the 995$r value >(with a value different from the 942$c) and the 995$h value >(3 possible values: Non-fiction, Fiction, Reference) >11 / Refresh the search for "livre" >12 / Check that the facet "Item types" has been updated and >that a facet "Collections" is shown (from the 995$h) and functional > >C. Check Elasticsearch indexing changes >13 / You cannot currently test Elasticsearch in a sandbox, >so, open a new k-t-d with Elasticsearch search engine (ktd --es7 up) >14 / Repeat operations 7 to 12 in that new instance >(277 results should be shown after the search in step 7, >instead of 419 for Zebra, this is normal) >--- > C4/Koha.pm | 10 ++- > .../searchengine/elasticsearch/mappings.yaml | 14 +++- > .../unimarc/biblios/biblio-koha-indexdefs.xml | 31 +++---- > .../biblios/biblio-zebra-indexdefs.xsl | 84 +++++++++---------- > .../mandatory/unimarc_framework_DEFAULT.yml | 60 +------------ > .../unimarc_sample_fastadd_framework.yml | 4 +- > .../data/mysql/mandatory/auth_val_cat.sql | 1 - > t/lib/TestBuilder.pm | 6 +- > 8 files changed, 82 insertions(+), 128 deletions(-) > >diff --git a/C4/Koha.pm b/C4/Koha.pm >index 39e543b71d..55de7a1ece 100644 >--- a/C4/Koha.pm >+++ b/C4/Koha.pm >@@ -349,14 +349,20 @@ sub getFacets { > sep => ', ', > }, > { >- idx => 'location', >+ idx => 'itype', >+ label => 'Item types', >+ tags => [ qw/ 995r 942c /], >+ sep => ', ', >+ }, >+ { >+ idx => 'location', > label => 'Location', > tags => [qw/ 995e /], > }, > { > idx => 'ccode', > label => 'Collections', >- tags => [qw / 099t 955h /], >+ tags => [ qw / 995h /], > } > ]; > >diff --git a/admin/searchengine/elasticsearch/mappings.yaml b/admin/searchengine/elasticsearch/mappings.yaml >index 139d66ffc4..cd1eeaabbe 100644 >--- a/admin/searchengine/elasticsearch/mappings.yaml >+++ b/admin/searchengine/elasticsearch/mappings.yaml >@@ -1351,7 +1351,7 @@ biblios: > sort: 1 > suggestible: '' > - facet: 1 >- marc_field: '9958' >+ marc_field: '995h' > marc_type: unimarc > sort: 1 > suggestible: '' >@@ -2255,6 +2255,11 @@ biblios: > marc_type: marc21 > sort: 1 > suggestible: '' >+ - facet: '' >+ marc_field: 942c >+ marc_type: unimarc >+ sort: 1 >+ suggestible: '' > opac: 1 > staff_client: 1 > type: '' >@@ -2274,7 +2279,7 @@ biblios: > sort: 1 > suggestible: '' > - facet: 1 >- marc_field: 200b >+ marc_field: 942c > marc_type: unimarc > sort: 1 > suggestible: '' >@@ -4403,6 +4408,11 @@ biblios: > marc_type: marc21 > sort: 1 > suggestible: '' >+ - facet: '' >+ marc_field: '9420' >+ marc_type: unimarc >+ sort: 1 >+ suggestible: '' > opac: 1 > staff_client: 1 > type: '' >diff --git a/etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml b/etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml >index 5425bc2ba6..55df730165 100644 >--- a/etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml >+++ b/etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml >@@ -39,10 +39,6 @@ > <target_index>Date/time-last-modified:n</target_index> > <target_index>Date/time-last-modified:y</target_index> > </index_subfields> >- <!--record.abs line 30: melm 099$t ccode:w--> >- <index_subfields tag="099" subfields="t"> >- <target_index>ccode:w</target_index> >- </index_subfields> > <!--010 : ISBN--> > <index_subfields tag="010" subfields="az"> > <target_index>ISBN:w</target_index> >@@ -103,15 +99,6 @@ > <target_index>EAN:w</target_index> > <target_index>Identifier-standard:w</target_index> > </index_subfields> >- <!--record.abs melm 200$b itemtype:w,itemtype:p,itype:w,itype:p,Material-type:w,Material-type:p--> >- <index_subfields tag="200" subfields="b"> >- <target_index>itemtype:w</target_index> >- <target_index>itemtype:p</target_index> >- <target_index>itype:w</target_index> >- <target_index>itype:p</target_index> >- <target_index>Material-type:w</target_index> >- <target_index>Material-type:p</target_index> >- </index_subfields> > <!--record.abs line 87: melm 995$r itemtype:w,itemtype:p,itype:w,itype:p--> > <index_subfields tag="995" subfields="r"> > <target_index>itemtype:w</target_index> >@@ -1446,12 +1433,15 @@ > <target_index>totalissues:n</target_index> > <target_index>totalissues:s</target_index> > </index_subfields> >+ <index_subfields tag="942" subfields="c"> >+ <target_index>itemtype:w</target_index> >+ <target_index>itemtype:p</target_index> >+ <target_index>itype:w</target_index> >+ <target_index>itype:p</target_index> >+ </index_subfields> > <index_subfields tag="999" subfields="x"> > <target_index>not-onloan-count:n</target_index> > </index_subfields> >- <facet tag="099" subfields="t"> >- <target_index>ccode:0</target_index> >- </facet> > <facet tag="225" subfields="a"> > <target_index>se:0</target_index> > </facet> >@@ -1488,6 +1478,9 @@ > <facet tag="702" subfields="ab"> > <target_index>au:0</target_index> > </facet> >+ <facet tag="942" subfields="c"> >+ <target_index>itype:0</target_index> >+ </facet> > <facet tag="995" subfields="b"> > <target_index>homebranch:0</target_index> > </facet> >@@ -1500,5 +1493,7 @@ > <facet tag="995" subfields="h"> > <target_index>ccode:0</target_index> > </facet> >- >-</kohaidx:index_defs> >+ <facet tag="995" subfields="r"> >+ <target_index>itype:0</target_index> >+ </facet> >+ </kohaidx:index_defs> >diff --git a/etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl b/etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl >index 22d902c461..b5ae8ce951 100644 >--- a/etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl >+++ b/etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl >@@ -87,13 +87,6 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > </z:index> > </xslo:if> > </xslo:for-each> >- <xslo:for-each select="marc:subfield"> >- <xslo:if test="contains('t', @code)"> >- <z:index name="ccode:w"> >- <xslo:value-of select="."/> >- </z:index> >- </xslo:if> >- </xslo:for-each> > </xslo:template> > <xslo:template mode="index_subfields" match="marc:datafield[@tag='010']"> > <xslo:for-each select="marc:subfield"> >@@ -217,36 +210,6 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > </xslo:if> > </xslo:for-each> > </xslo:template> >- <xslo:template mode="index_subfields" match="marc:datafield[@tag='200']"> >- <xslo:for-each select="marc:subfield"> >- <xslo:if test="contains('b', @code)"> >- <z:index name="itemtype:w itemtype:p itype:w itype:p Material-type:w Material-type:p"> >- <xslo:value-of select="."/> >- </z:index> >- </xslo:if> >- </xslo:for-each> >- <xslo:for-each select="marc:subfield"> >- <xslo:if test="contains('fg', @code)"> >- <z:index name="Author:w Author:p"> >- <xslo:value-of select="."/> >- </z:index> >- </xslo:if> >- </xslo:for-each> >- <xslo:for-each select="marc:subfield"> >- <xslo:if test="contains('aei', @code)"> >- <z:index name="Title:w Title:p Title:s Title-cover:w Title-cover:p Title-cover:s"> >- <xslo:value-of select="."/> >- </z:index> >- </xslo:if> >- </xslo:for-each> >- <xslo:for-each select="marc:subfield"> >- <xslo:if test="contains('cd', @code)"> >- <z:index name="Title:w Title:p"> >- <xslo:value-of select="."/> >- </z:index> >- </xslo:if> >- </xslo:for-each> >- </xslo:template> > <xslo:template mode="index_subfields" match="marc:datafield[@tag='995']"> > <xslo:for-each select="marc:subfield"> > <xslo:if test="contains('r', @code)"> >@@ -643,6 +606,29 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > </xslo:if> > </xslo:for-each> > </xslo:template> >+ <xslo:template mode="index_subfields" match="marc:datafield[@tag='200']"> >+ <xslo:for-each select="marc:subfield"> >+ <xslo:if test="contains('fg', @code)"> >+ <z:index name="Author:w Author:p"> >+ <xslo:value-of select="."/> >+ </z:index> >+ </xslo:if> >+ </xslo:for-each> >+ <xslo:for-each select="marc:subfield"> >+ <xslo:if test="contains('aei', @code)"> >+ <z:index name="Title:w Title:p Title:s Title-cover:w Title-cover:p Title-cover:s"> >+ <xslo:value-of select="."/> >+ </z:index> >+ </xslo:if> >+ </xslo:for-each> >+ <xslo:for-each select="marc:subfield"> >+ <xslo:if test="contains('cd', @code)"> >+ <z:index name="Title:w Title:p"> >+ <xslo:value-of select="."/> >+ </z:index> >+ </xslo:if> >+ </xslo:for-each> >+ </xslo:template> > <xslo:template mode="index_subfields" match="marc:datafield[@tag='700']"> > <xslo:for-each select="marc:subfield"> > <xslo:if test="contains('3', @code)"> >@@ -1882,6 +1868,13 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > </z:index> > </xslo:if> > </xslo:for-each> >+ <xslo:for-each select="marc:subfield"> >+ <xslo:if test="contains('c', @code)"> >+ <z:index name="itemtype:w itemtype:p itype:w itype:p"> >+ <xslo:value-of select="."/> >+ </z:index> >+ </xslo:if> >+ </xslo:for-each> > </xslo:template> > <xslo:template mode="index_subfields" match="marc:datafield[@tag='999']"> > <xslo:for-each select="marc:subfield"> >@@ -2542,13 +2535,6 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > <xslo:value-of select="normalize-space($raw_heading)"/> > </z:index> > </xslo:template> >- <xslo:template mode="index_facets" match="marc:datafield[@tag='099']"> >- <xslo:if test="not(@ind1='z')"> >- <z:index name="ccode:0"> >- <xslo:value-of select="marc:subfield[@code='t']"/> >- </z:index> >- </xslo:if> >- </xslo:template> > <xslo:template mode="index_facets" match="marc:datafield[@tag='225']"> > <xslo:if test="not(@ind1='z')"> > <z:index name="se:0"> >@@ -2661,6 +2647,13 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > </z:index> > </xslo:if> > </xslo:template> >+ <xslo:template mode="index_facets" match="marc:datafield[@tag='942']"> >+ <xslo:if test="not(@ind1='z')"> >+ <z:index name="itype:0"> >+ <xslo:value-of select="marc:subfield[@code='c']"/> >+ </z:index> >+ </xslo:if> >+ </xslo:template> > <xslo:template mode="index_facets" match="marc:datafield[@tag='995']"> > <xslo:if test="not(@ind1='z')"> > <z:index name="homebranch:0"> >@@ -2675,6 +2668,9 @@ definition file (probably something like {biblio,authority}-koha-indexdefs.xml) > <z:index name="ccode:0"> > <xslo:value-of select="marc:subfield[@code='h']"/> > </z:index> >+ <z:index name="itype:0"> >+ <xslo:value-of select="marc:subfield[@code='r']"/> >+ </z:index> > </xslo:if> > </xslo:template> > <xslo:template mode="index_all" match="text()"> >diff --git a/installer/data/mysql/en/marcflavour/unimarc/mandatory/unimarc_framework_DEFAULT.yml b/installer/data/mysql/en/marcflavour/unimarc/mandatory/unimarc_framework_DEFAULT.yml >index afc80b8425..d088e123fe 100644 >--- a/installer/data/mysql/en/marcflavour/unimarc/mandatory/unimarc_framework_DEFAULT.yml >+++ b/installer/data/mysql/en/marcflavour/unimarc/mandatory/unimarc_framework_DEFAULT.yml >@@ -423,8 +423,8 @@ tables: > multiline: [] > rows: > - tagfield: "099" >- liblibrarian: "local dates" >- libopac: "" >+ liblibrarian: "Local dates (Koha)" >+ libopac: "Local dates (Koha)" > repeatable: 0 > mandatory: 0 > authorised_value: >@@ -470,60 +470,6 @@ tables: > link: "" > defaultvalue: > >- - tagfield: "099" >- tagsubfield: "s" >- liblibrarian: "Serial" >- libopac: "Serial" >- repeatable: 0 >- mandatory: 0 >- kohafield: biblio.serial >- tab: 0 >- authorised_value: >- authtypecode: "" >- value_builder: "" >- isurl: 0 >- hidden: 7 >- frameworkcode: "" >- seealso: "" >- link: "" >- defaultvalue: >- >- - tagfield: "099" >- tagsubfield: "t" >- liblibrarian: "Koha item type" >- libopac: "Koha item type" >- repeatable: 0 >- mandatory: 1 >- kohafield: biblioitems.itemtype >- tab: 0 >- authorised_value: TYPEDOC >- authtypecode: "" >- value_builder: "" >- isurl: 0 >- hidden: 0 >- frameworkcode: "" >- seealso: "" >- link: "" >- defaultvalue: >- >- - tagfield: "099" >- tagsubfield: "x" >- liblibrarian: "Number of issues" >- libopac: "Number of issues" >- repeatable: 0 >- mandatory: 0 >- kohafield: biblioitems.totalissues >- tab: 0 >- authorised_value: >- authtypecode: "" >- value_builder: "" >- isurl: 0 >- hidden: 6 >- frameworkcode: "" >- seealso: "" >- link: "" >- defaultvalue: "" >- > - marc_tag_structure: > translatable: [ liblibrarian, libopac ] > multiline: [] >@@ -532,7 +478,7 @@ tables: > liblibrarian: "ADDED ENTRY ELEMENTS (KOHA)" > libopac: "ADDED ENTRY ELEMENTS (KOHA)" > repeatable: 0 >- mandatory: 0 >+ mandatory: 1 > authorised_value: > frameworkcode: "" > >diff --git a/installer/data/mysql/en/marcflavour/unimarc/optional/unimarc_sample_fastadd_framework.yml b/installer/data/mysql/en/marcflavour/unimarc/optional/unimarc_sample_fastadd_framework.yml >index 0653a51e76..c729b60363 100644 >--- a/installer/data/mysql/en/marcflavour/unimarc/optional/unimarc_sample_fastadd_framework.yml >+++ b/installer/data/mysql/en/marcflavour/unimarc/optional/unimarc_sample_fastadd_framework.yml >@@ -38,11 +38,11 @@ sql_statements: > SELECT tagfield, liblibrarian, libopac, repeatable, mandatory, authorised_value, 'FA' > FROM marc_tag_structure > WHERE frameworkcode = '' AND >- tagfield IN ('000', '001', '090', '099', '100', '200', '205', '210', '225', '995');" >+ tagfield IN ('000', '001', '090', '099', '100', '200', '205', '210', '225', '942', '995');" > > > - "INSERT IGNORE INTO marc_subfield_structure (tagfield, tagsubfield, liblibrarian, libopac, repeatable, mandatory, kohafield, tab, authorised_value, authtypecode, value_builder, isurl, hidden, frameworkcode, seealso, link, defaultvalue) > SELECT tagfield, tagsubfield, liblibrarian, libopac, repeatable, mandatory, kohafield, tab, authorised_value, authtypecode, value_builder, isurl, hidden, 'FA', seealso, link, defaultvalue > FROM marc_subfield_structure > WHERE frameworkcode = '' AND >- tagfield IN ('000', '001', '090', '099', '100', '200', '205', '210', '225', '995');" >+ tagfield IN ('000', '001', '090', '099', '100', '200', '205', '210', '225', '942', '995');" >diff --git a/installer/data/mysql/mandatory/auth_val_cat.sql b/installer/data/mysql/mandatory/auth_val_cat.sql >index c1428e4a3c..e15875ca9d 100644 >--- a/installer/data/mysql/mandatory/auth_val_cat.sql >+++ b/installer/data/mysql/mandatory/auth_val_cat.sql >@@ -57,7 +57,6 @@ INSERT IGNORE INTO authorised_value_categories( category_name ) VALUES > ('ETAT'), > ('CAND'), > ('COUNTRY'), >- ('TYPEDOC'), > ('qualif'); > > -- For Housebound >diff --git a/t/lib/TestBuilder.pm b/t/lib/TestBuilder.pm >index a0490c6531..1ad15d3639 100644 >--- a/t/lib/TestBuilder.pm >+++ b/t/lib/TestBuilder.pm >@@ -185,8 +185,10 @@ sub build_sample_biblio { > MARC::Field->new( $tag, ' ', ' ', $subfield => $author ), > ); > >- ( $tag, $subfield ) = $marcflavour eq 'UNIMARC' ? ( '099', 't' ) : ( 942, 'c' ); >- $record->append_fields( MARC::Field->new( $tag, ' ', ' ', $subfield => $itemtype ) ); >+ ( $tag, $subfield ) = $marcflavour eq 'UNIMARC' ? ( 942, 'c' ) : ( 942, 'c' ); >+ $record->append_fields( >+ MARC::Field->new( $tag, ' ', ' ', $subfield => $itemtype ), >+ ); > > my ($biblio_id) = C4::Biblio::AddBiblio( $record, $frameworkcode ); > return Koha::Biblios->find($biblio_id); >-- >2.39.5
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 38336
:
178126
|
179007
|
182471