Bug 24517 - Zebra: date-entered-on-file misses 6th position
Summary: Zebra: date-entered-on-file misses 6th position
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: Main
Hardware: All All
: P5 - low minor (vote)
Assignee: David Cook
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-01-27 07:59 UTC by Marcel de Rooy
Modified: 2023-12-28 20:46 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
This patch fixes the date-entered-on-file index so that it correctly uses all 6 characters instead of the 5 character it has used the last 11 years. Note: For this patch to have effect, Zebra must be re-indexed.
Version(s) released in:
23.11.00,23.05.02,22.11.08


Attachments
Bug 24517: Add missing 6th position to date-entered-on-file index (1.64 KB, patch)
2023-06-07 03:27 UTC, David Cook
Details | Diff | Splinter Review
Bug 24517: Add missing 6th position to date-entered-on-file index (XSLT) (1.26 KB, patch)
2023-06-07 03:27 UTC, David Cook
Details | Diff | Splinter Review
Bug 24517: Add missing 6th position to date-entered-on-file index (1.70 KB, patch)
2023-06-08 22:54 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 24517: Add missing 6th position to date-entered-on-file index (XSLT) (1.32 KB, patch)
2023-06-08 22:55 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 24517: Add missing 6th position to date-entered-on-file index (2.84 KB, patch)
2023-06-23 08:02 UTC, Marcel de Rooy
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Marcel de Rooy 2020-01-27 07:59:44 UTC
Date entered on file is in MARC21 tag 008 position 00 to 05 (or 1-6 for normal humans). In Koha only the first 5 digits are indexed with Zebra. Does not really facilitate searching ;)

  <index_control_field tag="008" offset="0" length="5">
    <target_index>date-entered-on-file:n</target_index>
  </index_control_field>
  <index_control_field tag="008" offset="0" length="5">
    <target_index>date-entered-on-file:s</target_index>
  </index_control_field>

  <xslo:template match="marc:controlfield[@tag='008']">
    <z:index name="date-entered-on-file:n">
      <xslo:value-of select="substring(., 1, 5)"/>
    </z:index>
    <z:index name="date-entered-on-file:s">
      <xslo:value-of select="substring(., 1, 5)"/>
    </z:index>

Elasticsearch appears to do a better job:

  date-entered-on-file:
    label: date-entered-on-file
    mappings:
      - facet: ''
        marc_field: '008_/0-5'
        marc_type: marc21
        sort: ~
        suggestible: ''
      - facet: ''
        marc_field: '008_/0-5'
        marc_type: normarc
        sort: ~
        suggestible: ''
Comment 1 David Cook 2023-06-07 03:09:08 UTC
Just bumped into this one. Fun times 🤣
Comment 2 David Cook 2023-06-07 03:12:25 UTC
Couldn't recall if we just update the XML or also the XSLT, so I'll do two separate patches.

This is how I generated the XSLT though:

xsltproc ./etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml > ./etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
Comment 3 David Cook 2023-06-07 03:27:09 UTC
Test plan:

Before applying patch...

1. yaz-client unix:/var/run/koha/kohadev/bibliosocket
2. base biblios
3. querytype ccl2rpn
4. set_cclfile /etc/koha/zebradb/ccl.properties
5. find date-entered-on-file,st-numeric < 23000
6. format xml
7. elements zebra::snippet
8. show 1
9. Note that only 5 digits appear in the "date-entered-on-file" snippet (e.g. 12082)

--
Apply the patch

0a. cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
0b. koha-rebuild-zebra -b -f -v kohadev

1. yaz-client unix:/var/run/koha/kohadev/bibliosocket
2. base biblios
3. querytype ccl2rpn
4. set_cclfile /etc/koha/zebradb/ccl.properties
5. find date-entered-on-file,st-numeric < 230000
6. format xml
7. elements zebra::snippet
8. show 1
9. Note that 6 digits appear in the "date-entered-on-file" snippet (e.g. 120829)
Comment 4 David Cook 2023-06-07 03:27:38 UTC
Created attachment 152071 [details] [review]
Bug 24517: Add missing 6th position to date-entered-on-file index
Comment 5 David Cook 2023-06-07 03:27:41 UTC
Created attachment 152072 [details] [review]
Bug 24517: Add missing 6th position to date-entered-on-file index (XSLT)
Comment 6 Katrin Fischer 2023-06-08 22:54:58 UTC
Created attachment 152223 [details] [review]
Bug 24517: Add missing 6th position to date-entered-on-file index

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 7 Katrin Fischer 2023-06-08 22:55:02 UTC
Created attachment 152224 [details] [review]
Bug 24517: Add missing 6th position to date-entered-on-file index (XSLT)

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 8 Marcel de Rooy 2023-06-23 08:02:14 UTC
-  <index_control_field tag="008" offset="0" length="5">
+  <index_control_field tag="008" offset="0" length="6">

-      <xslo:value-of select="substring(., 1, 5)"/>
+      <xslo:value-of select="substring(., 1, 6)"/>

This is ok. Just confusing that XSLT substring starts with 1 while we use offset with zero..
Comment 9 Marcel de Rooy 2023-06-23 08:02:52 UTC
Created attachment 152621 [details] [review]
Bug 24517: Add missing 6th position to date-entered-on-file index

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Adding the biblio-zebra-indexdefs.xsl on same patch (as should
be generated with xsltproc).
Comment 10 Tomás Cohen Arazi 2023-06-23 12:45:49 UTC
Pushed to master for 23.11.

Nice work everyone, thanks!
Comment 11 Martin Renvoize 2023-07-17 12:45:22 UTC
Thanks for all the hard work!

Pushed to 23.05.x for the next release
Comment 12 Pedro Amorim 2023-07-18 10:49:56 UTC
Nice work everyone!

Pushed to 22.11.x for next release