Bug 15704

Summary: The 264 index should be split by subfield to match how 260 is indexed
Product: Koha Reporter: Nicole C. Engard <nengard>
Component: SearchingAssignee: Nick Clemens <nick>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: minor    
Priority: P5 - low CC: andrewfh, barton, charles.farmer, felicia.martin, fridolin.somers, hector.hecaxmmx, katrin.fischer, kelly, lucas, margaret, marjorie.barry-vila, martin.renvoize, nick
Version: master   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
19.11.00,19.05.03
Bug Depends on: 14198    
Bug Blocks:    
Attachments: Bug 15704 - Split up zebra indexing of RDA publisher information
Bug 15704: Split up Zebra indexing of RDA 264 information

Description Nicole C. Engard 2016-01-29 15:58:26 UTC
Right now the 260 index look like this: 

  <!--record.abs line 119: melm 260$a      pl:w,pl:p-->
  <index_subfields tag="260" subfields="a">
    <target_index>pl:w</target_index>
    <target_index>pl:p</target_index>
  </index_subfields>
  <!--record.abs line 120: melm 260$b      Publisher:w,Publisher:p-->
  <index_subfields tag="260" subfields="b">
    <target_index>Publisher:w</target_index>
    <target_index>Publisher:p</target_index>
  </index_subfields>
  <!--record.abs line 121: melm 260$c      copydate,copydate:s-->
  <index_subfields tag="260" subfields="c">
    <target_index>copydate:w</target_index>
    <target_index>copydate:s</target_index>
  </index_subfields>
  <!--record.abs line 122: melm 260        pl-->
  <index_data_field tag="260">
    <target_index>pl:w</target_index>
    <target_index>Provider:w</target_index>
  </index_data_field>

The 264 only indexes the entire thing. 

  <!--Index for RDA 264 field-->
  <index_data_field tag="264">
    <target_index>Provider:w</target_index>
  </index_data_field>

We should break the 264 up for better/more accurate searching.
Comment 1 Héctor Eduardo Castro Avalos 2016-01-31 04:37:07 UTC
Hi Katrin 

Split-up the index like 260 was my first intention when I added the new index "provider" also was splitted up in 'Provider-place', and 'Provider-date'.

NOTE: Clarified that the latter two do not exist in the documentation.
<http://www.loc.gov/z3950/agency/defns/bib1.html>

If we want to collect all indexes in Provider -this is the true intention of the index, not just collect 260, 264 but also 270, 257- works if you change etc/zebradb/ccl.properties as follow

Provider 1=1225
pv Provider Place-publication Publisher copydate

the exmaple above follows yaz documentation at <http://www.indexdata.com/yaz/doc/tools.html#CCL> in section 1.2.2.2 Qualifier alias.

This at least is not more accurate but will launch more search results

Please let me know which solution do you prefer or/also I can attach two patches to see which is most suitable.

Regards
Comment 2 Héctor Eduardo Castro Avalos 2016-02-02 16:09:40 UTC
Hi Nicole

At first glance I thought it was Katrin who file this bug, but I was wrong.

What do you think about comment 1, or could you propose something at this point?
Comment 3 Nick Clemens 2017-12-18 15:47:16 UTC
Created attachment 69852 [details] [review]
Bug 15704 - Split up zebra indexing of RDA  publisher information

To test:
1 - Add a record with a unique publisher "Supercalifragilistic" in the
264 b field
2 - Search for the value
3 - Record not found
4 - Apply patch (may need ot copy the .xml file into koha install)
5 - Reindex all the things
6 - Search for the value
7 - Success!
Comment 4 Charles Farmer 2018-06-15 19:32:35 UTC
Using the most basic record with the 'Default framework', I can't reproduce the issue of not finding a record from its publisher.

Here's what I did:

* Since the 264's weren't visible by default in the editor, I went ahead and made them show up by changing the properties of admin/biblio_framework.pl. Otherwise, I had no way of adding content to those fields
* I created a new record, filled the info.
* * I also made sure to fill the other fields of 264 with garbage data, just in case
* Rebuilt zebra
* Searched for the string I added in 264b: supercalifragilisticexpialidocious
* I'm redirected to the new record's page

Am I doing something wrong here? Why is it showing up on my master?
Comment 5 Katrin Fischer 2018-07-14 12:33:28 UTC
>Am I doing something wrong here? Why is it showing up on my master?

How did you search for it? Since we switched to dom indexing you can always search everything in the record within the keyword index, so it should always show up.

There also was bug 14198. 

The interesting question is if you can find it in the specific indexes for place, year and publisher. :)
Comment 6 Cab Vinton 2018-08-17 23:59:58 UTC
Not sure this is a bug.

Publisher in the 264$b appears to be searchable via the publisher index even without the patch applied (e.g., search.pl?idx=pb&q=superkapow). Also searchable via a plain keyword search.
Comment 7 Andrew Fuerste-Henry 2019-01-11 16:58:26 UTC
Hi Cab! Are you sure the record used in your example search didn't also have that data in some field that *is* indexed as publisher? I just tested this on a library migrating to 18.05.05 and records that have publisher data only in the 264 don't come up in a publisher search.
Comment 8 Cab Vinton 2019-01-12 01:50:47 UTC
You may be right, Andrew. Not sure how I was testing back before the fall -- lost in the mists of time -- but I most likely did overlook something ...
Comment 9 Katrin Fischer 2019-05-11 16:52:54 UTC
Created attachment 89615 [details] [review]
Bug 15704: Split up Zebra indexing of RDA 264 information

To test:
1 - Add a record with a unique publisher "Supercalifragilistic" in the
264 b field
2 - Search for the value
3 - Record not found
4 - Apply patch (may need ot copy the .xml file into koha install)
5 - Reindex all the things
6 - Search for the value
7 - Success!

Signed-off-by: Felicia Martin <felicia.martin@dncr.nh.gov>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 10 Katrin Fischer 2019-05-11 16:54:22 UTC
I've added Felicia's sign off line as per the change history and the "Who signed the patch off" field in bugzilla.
Comment 11 Martin Renvoize 2019-06-24 14:47:45 UTC
Nice work!

Pushed to master for 19.11.00
Comment 12 Fridolin Somers 2019-07-24 07:05:08 UTC
Pushed to 19.05.x for 19.05.03
Comment 13 Lucas Gass 2019-07-25 20:35:39 UTC
this is searchable without the patch for me, im not backporting to 18.11.x
Comment 14 Katrin Fischer 2019-10-29 15:56:58 UTC
(In reply to Lucas Gass from comment #13)
> this is searchable without the patch for me, im not backporting to 18.11.x

It is not, not in the correct indexes... If you search in Advanced search for publication year or publisher, it won't be found. It's only searchable via Provider or keyword index.
Comment 15 Katrin Fischer 2020-01-11 21:34:49 UTC
*** Bug 18892 has been marked as a duplicate of this bug. ***