Bug 36724 - Add cartographic mathematical data to Elasticsearch index mappings
Summary: Add cartographic mathematical data to Elasticsearch index mappings
Status: Signed Off
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement
Assignee: Aleisha Amohia
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-04-30 04:56 UTC by Aleisha Amohia
Modified: 2025-08-15 03:11 UTC (History)
5 users (show)

See Also:
GIT URL:
Change sponsored?: Sponsored
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Values in 255$a and 255$b in MARC21 bibliographic records are now indexed and searchable in Elasticsearch.
Version(s) released in:
Circulation function:


Attachments
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings (1.81 KB, patch)
2024-06-05 04:51 UTC, Danyon Sewell
Details | Diff | Splinter Review
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings (1.76 KB, patch)
2024-06-05 05:16 UTC, Danyon Sewell
Details | Diff | Splinter Review
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings (1.68 KB, patch)
2024-06-12 01:59 UTC, Danyon Sewell
Details | Diff | Splinter Review
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings (1.51 KB, patch)
2025-08-07 23:39 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings (1.55 KB, patch)
2025-08-15 03:11 UTC, David Nind
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Aleisha Amohia 2024-04-30 04:56:58 UTC
Search whole 255 field https://www.loc.gov/marc/bibliographic/bd255.html

Test with values like 

* Scale 1:22,000,000
* Conic proj.
Comment 1 Danyon Sewell 2024-06-05 04:51:50 UTC
Created attachment 167426 [details] [review]
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings

Have written a patch to allow cartographic mathematical data to be searched using the default keyword search index.

Testing plan:

1. Start up KTD with Elasticsearch: ktd --es8 up
2. Apply the patch and restart everything (restart_all).
3. Reset the mappings: Administration > Catalog > Search engine configuration (Elasticsearch) > Reset mappings (at the bottom of the page)
4. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev
5. Alternative to steps 3 and 4: reset_all
6. Update the visibility for 255$a & 255$b in the default framework so that it is visible in the Editor.
7. Add a new record using the default framework and put a unique term in both field 255$a and 255$b.
8. Using the basic keyword search, search for the terms you put in step 7.
Comment 2 Aleisha Amohia 2024-06-05 05:02:47 UTC
Hey Danyon, this looks great and is almost there. Just a few changes to make: 

1. Rename the patch to:

Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings

(just moving the colon)

2. the `label` of the index needs to have no spaces, it's the name of the index. For eg:

cartographic-mathematical-data

3. When a mapping uses multiple subfields from the same tag, you can consolidate them like this:

marc_field: '255ab'

Can you please make those changes and reattach the patch?
Comment 3 Danyon Sewell 2024-06-05 05:16:07 UTC
Created attachment 167428 [details] [review]
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings

ah my mistake, thanks for the pointers.

Have gone ahead and made the requested changes.

Testing plan:

1. Start up KTD with Elasticsearch: ktd --es8 up
2. Apply the patch and restart everything (restart_all).
3. Reset the mappings: Administration > Catalog > Search engine configuration (Elasticsearch) > Reset mappings (at the bottom of the page)
4. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev
5. Alternative to steps 3 and 4: reset_all
6. Update the visibility for 255$a & 255$b in the default framework so that it is visible in the Editor.
7. Add a new record using the default framework and put a unique term in both field 255$a and 255$b.
8. Using the basic keyword search, search for the terms you put in step 7.
Comment 4 David Nind 2024-06-08 01:26:28 UTC
There is a sha1/fake ancestor error when applying the patch:

git bz apply 36724

Bug 36724 - Add cartographic mathematical data to Elasticsearch index mappings

167428 - Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings
error: sha1 information is lacking or useless (admin/searchengine/elasticsearch/mappings.yaml).
error: could not build fake ancestor
Patch failed at 0001 Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings
Comment 5 Danyon Sewell 2024-06-12 01:59:46 UTC
Created attachment 167647 [details] [review]
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings

Have reattached the patch hopefully this time without any sha1/fake ancestor errors.


Testing plan:

1. Start up KTD with Elasticsearch: ktd --es8 up
2. Apply the patch and restart everything (restart_all).
3. Reset the mappings: Administration > Catalog > Search engine configuration (Elasticsearch) > Reset mappings (at the bottom of the page)
4. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev
5. Alternative to steps 3 and 4: reset_all
6. Update the visibility for 255$a & 255$b in the default framework so that it is visible in the Editor.
7. Add a new record using the default framework and put a unique term in both field 255$a and 255$b.
8. Using the basic keyword search, search for the terms you put in step 7.
Comment 6 Laurence Rault 2024-09-27 19:46:25 UTC
Testing plan works as expected.

However, I don't understand why the index cartographic-mathematical-data is added as a new label in "abstract" search field in the mappings.yaml file instead of being apart : 

biblios:
  abstract:
    label: abstract
    mandatory: ~
    mappings:
    - facet: ''
      marc_field: '520'
      marc_type: marc21
      sort: 1
      suggestible: ''
    opac: 1
    staff_client: 1
    type: ''
    label: cartographic-mathematical-data
    mandatory: ~
    mappings:
    - facet: ''
      marc_field: '255ab'
      marc_type: marc21
      sort: 1
      suggestible: ''
    opac: 1
    staff_client: 1
    type: ''
Comment 7 Janusz Kaczmarek 2025-08-07 18:13:08 UTC
IMO the patch lacks the name of the new search-field. The insert starts from label: while it should add a new search field one level above. 

But, since we already have the 'map-scale' search-field, the indexing of 255 field could be added there.

What do you think?
Comment 8 Aleisha Amohia 2025-08-07 23:39:14 UTC
Created attachment 185255 [details] [review]
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings

Testing plan:

1. Start up KTD with Elasticsearch: ktd --es8 up
2. Apply the patch and restart everything (restart_all).
3. Reindex and reset mappings: koha-elasticsearch --rebuild -r -b -a kohadev
4. Go to Koha administration -> MARC bibliographic frameworks and edit the Default framework. Edit the subfields for the 255 tag. Update the visibility for 255$a & 255$b so that it is visible in the Editor.
5. Add a new record using the default framework and put a unique term in both field 255$a and 255$b.
6. Using the basic keyword search, search for the terms you put in step 7. Confirm your new record is returned in the results.

Sponsored-by: Education Services Australia SCIS
Comment 9 David Nind 2025-08-15 03:11:49 UTC
Created attachment 185435 [details] [review]
Bug 36724: Add cartographic mathematical data to Elasticsearch index mappings

Testing plan:

1. Start up KTD with Elasticsearch: ktd --es8 up
2. Apply the patch and restart everything (restart_all).
3. Reindex and reset mappings: koha-elasticsearch --rebuild -r -b -a kohadev
4. Go to Koha administration -> MARC bibliographic frameworks and edit the Default framework. Edit the subfields for the 255 tag. Update the visibility for 255$a & 255$b so that it is visible in the Editor.
5. Add a new record using the default framework and put a unique term in both field 255$a and 255$b.
6. Using the basic keyword search, search for the terms you put in step 7. Confirm your new record is returned in the results.

Sponsored-by: Education Services Australia SCIS
Signed-off-by: David Nind <david@davidnind.com>