Bug 31693 - Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21)
Summary: Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21)
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Caroline Cyr La Rose
QA Contact: Kyle M Hall
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-10-05 16:20 UTC by Caroline Cyr La Rose
Modified: 2023-12-28 20:43 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
This enhancement removes fields from the See-from index that don't exist in MARC21. The existing fields can be found here: https://www.loc.gov/marc/authority/ad5xx.html
Version(s) released in:
22.11.00


Attachments
Bug 31693 Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21) (3.14 KB, patch)
2022-10-05 16:23 UTC, Caroline Cyr La Rose
Details | Diff | Splinter Review
Bug 31693 Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21) (3.14 KB, patch)
2022-10-05 16:24 UTC, Caroline Cyr La Rose
Details | Diff | Splinter Review
Bug 31693 Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21) (3.19 KB, patch)
2022-10-10 19:17 UTC, David Nind
Details | Diff | Splinter Review
Bug 31693 Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21) (3.25 KB, patch)
2022-10-21 11:39 UTC, Kyle M Hall
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Caroline Cyr La Rose 2022-10-05 16:20:58 UTC
The See-also-from index for MARC21 indexes fields that don't exist in the MARC21 format.

The existing fields can be found here: https://www.loc.gov/marc/authority/ad5xx.html

Here are the fields currently indexed that are not part of MARC21.

- 511$b
- 511$m
- 511$o
- 511$r
- 530$b
- 530$c
- 530$e
- 547$b
- 547$e
- 547$f
- 547$h
- 547$j
- 547$k
- 547$l
- 547$m
- 547$n
- 547$o
- 547$p
- 547$q
- 547$r
- 547$s
- 547$t
- 548$b
- 548$c
- 548$d
- 548$e
- 548$f
- 548$g
- 548$h
- 548$k
- 548$l
- 548$m
- 548$n
- 548$o
- 548$p
- 548$r
- 548$s
- 548$t
- 550$c
- 550$d
- 550$e
- 550$f
- 550$h
- 550$j
- 550$k
- 550$l
- 550$m
- 550$n
- 550$o
- 550$p
- 550$q
- 550$r
- 550$s
- 550$t
- 551$b
- 551$c
- 551$d
- 551$e
- 551$f
- 551$h
- 551$k
- 551$l
- 551$m
- 551$n
- 551$o
- 551$p
- 551$r
- 551$s
- 551$t
- 555$b
- 555$c
- 555$d
- 555$e
- 555$f
- 555$g
- 555$h
- 555$j
- 555$k
- 555$l
- 555$m
- 555$n
- 555$o
- 555$p
- 555$q
- 555$r
- 555$s
- 555$t
- 562$b
- 562$c
- 562$d
- 562$e
- 562$f
- 562$g
- 562$h
- 562$k
- 562$l
- 562$m
- 562$n
- 562$o
- 562$p
- 562$r
- 562$s
- 562$t

Also, the mapping for 511 is written differently than other mappings, with the a outside the parentheses. 

511a(bcdefghjklmnopqrstvxyz)
Comment 1 Caroline Cyr La Rose 2022-10-05 16:23:00 UTC
Created attachment 141389 [details] [review]
Bug 31693 Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21)

This patch removes fields from the See-from index that don't exist in
MARC21.

The existing fields can be found here: https://www.loc.gov/marc/authority/ad5xx.html

The following fields are removed:

- 511$b
- 511$m
- 511$o
- 511$r
- 530$b
- 530$c
- 530$e
- 547$b
- 547$e
- 547$f
- 547$h
- 547$j
- 547$k
- 547$l
- 547$m
- 547$n
- 547$o
- 547$p
- 547$q
- 547$r
- 547$s
- 547$t
- 548$b
- 548$c
- 548$d
- 548$e
- 548$f
- 548$g
- 548$h
- 548$k
- 548$l
- 548$m
- 548$n
- 548$o
- 548$p
- 548$r
- 548$s
- 548$t
- 550$c
- 550$d
- 550$e
- 550$f
- 550$h
- 550$j
- 550$k
- 550$l
- 550$m
- 550$n
- 550$o
- 550$p
- 550$q
- 550$r
- 550$s
- 550$t
- 551$b
- 551$c
- 551$d
- 551$e
- 551$f
- 551$h
- 551$k
- 551$l
- 551$m
- 551$n
- 551$o
- 551$p
- 551$r
- 551$s
- 551$t
- 555$b
- 555$c
- 555$d
- 555$e
- 555$f
- 555$g
- 555$h
- 555$j
- 555$k
- 555$l
- 555$m
- 555$n
- 555$o
- 555$p
- 555$q
- 555$r
- 555$s
- 555$t
- 562$b
- 562$c
- 562$d
- 562$e
- 562$f
- 562$g
- 562$h
- 562$k
- 562$l
- 562$m
- 562$n
- 562$o
- 562$p
- 562$r
- 562$s
- 562$t

Furthermore, the format of the mapping for 511 has been corrected.
Comment 2 Caroline Cyr La Rose 2022-10-05 16:24:30 UTC
Created attachment 141391 [details] [review]
Bug 31693 Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21)

This patch removes fields from the See-also-from index that don't exist in
MARC21.

The existing fields can be found here: https://www.loc.gov/marc/authority/ad5xx.html

The following fields are removed:

- 511$b
- 511$m
- 511$o
- 511$r
- 530$b
- 530$c
- 530$e
- 547$b
- 547$e
- 547$f
- 547$h
- 547$j
- 547$k
- 547$l
- 547$m
- 547$n
- 547$o
- 547$p
- 547$q
- 547$r
- 547$s
- 547$t
- 548$b
- 548$c
- 548$d
- 548$e
- 548$f
- 548$g
- 548$h
- 548$k
- 548$l
- 548$m
- 548$n
- 548$o
- 548$p
- 548$r
- 548$s
- 548$t
- 550$c
- 550$d
- 550$e
- 550$f
- 550$h
- 550$j
- 550$k
- 550$l
- 550$m
- 550$n
- 550$o
- 550$p
- 550$q
- 550$r
- 550$s
- 550$t
- 551$b
- 551$c
- 551$d
- 551$e
- 551$f
- 551$h
- 551$k
- 551$l
- 551$m
- 551$n
- 551$o
- 551$p
- 551$r
- 551$s
- 551$t
- 555$b
- 555$c
- 555$d
- 555$e
- 555$f
- 555$g
- 555$h
- 555$j
- 555$k
- 555$l
- 555$m
- 555$n
- 555$o
- 555$p
- 555$q
- 555$r
- 555$s
- 555$t
- 562$b
- 562$c
- 562$d
- 562$e
- 562$f
- 562$g
- 562$h
- 562$k
- 562$l
- 562$m
- 562$n
- 562$o
- 562$p
- 562$r
- 562$s
- 562$t

Furthermore, the format of the mapping for 511 has been corrected.
Comment 3 Caroline Cyr La Rose 2022-10-05 16:25:09 UTC
I corrected the index name in my commit message...
Comment 4 David Nind 2022-10-10 19:17:49 UTC
Created attachment 141584 [details] [review]
Bug 31693 Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21)

This patch removes fields from the See-from index that don't exist in
MARC21.

The existing fields can be found here: https://www.loc.gov/marc/authority/ad5xx.html

The following fields are removed:

- 511$b
- 511$m
- 511$o
- 511$r
- 530$b
- 530$c
- 530$e
- 547$b
- 547$e
- 547$f
- 547$h
- 547$j
- 547$k
- 547$l
- 547$m
- 547$n
- 547$o
- 547$p
- 547$q
- 547$r
- 547$s
- 547$t
- 548$b
- 548$c
- 548$d
- 548$e
- 548$f
- 548$g
- 548$h
- 548$k
- 548$l
- 548$m
- 548$n
- 548$o
- 548$p
- 548$r
- 548$s
- 548$t
- 550$c
- 550$d
- 550$e
- 550$f
- 550$h
- 550$j
- 550$k
- 550$l
- 550$m
- 550$n
- 550$o
- 550$p
- 550$q
- 550$r
- 550$s
- 550$t
- 551$b
- 551$c
- 551$d
- 551$e
- 551$f
- 551$h
- 551$k
- 551$l
- 551$m
- 551$n
- 551$o
- 551$p
- 551$r
- 551$s
- 551$t
- 555$b
- 555$c
- 555$d
- 555$e
- 555$f
- 555$g
- 555$h
- 555$j
- 555$k
- 555$l
- 555$m
- 555$n
- 555$o
- 555$p
- 555$q
- 555$r
- 555$s
- 555$t
- 562$b
- 562$c
- 562$d
- 562$e
- 562$f
- 562$g
- 562$h
- 562$k
- 562$l
- 562$m
- 562$n
- 562$o
- 562$p
- 562$r
- 562$s
- 562$t

Furthermore, the format of the mapping for 511 has been corrected.

Signed-off-by: David Nind <david@davidnind.com>
Comment 5 David Nind 2022-10-10 19:19:15 UTC
"Tested" by comparing the list of fields removed with the Library of Congress definitions to make sure they don't exist.
Comment 6 Kyle M Hall 2022-10-21 11:39:04 UTC
Created attachment 142336 [details] [review]
Bug 31693 Remove non-existent fields from the See-also-from index (Elasticsearch, MARC21)

This patch removes fields from the See-from index that don't exist in
MARC21.

The existing fields can be found here: https://www.loc.gov/marc/authority/ad5xx.html

The following fields are removed:

- 511$b
- 511$m
- 511$o
- 511$r
- 530$b
- 530$c
- 530$e
- 547$b
- 547$e
- 547$f
- 547$h
- 547$j
- 547$k
- 547$l
- 547$m
- 547$n
- 547$o
- 547$p
- 547$q
- 547$r
- 547$s
- 547$t
- 548$b
- 548$c
- 548$d
- 548$e
- 548$f
- 548$g
- 548$h
- 548$k
- 548$l
- 548$m
- 548$n
- 548$o
- 548$p
- 548$r
- 548$s
- 548$t
- 550$c
- 550$d
- 550$e
- 550$f
- 550$h
- 550$j
- 550$k
- 550$l
- 550$m
- 550$n
- 550$o
- 550$p
- 550$q
- 550$r
- 550$s
- 550$t
- 551$b
- 551$c
- 551$d
- 551$e
- 551$f
- 551$h
- 551$k
- 551$l
- 551$m
- 551$n
- 551$o
- 551$p
- 551$r
- 551$s
- 551$t
- 555$b
- 555$c
- 555$d
- 555$e
- 555$f
- 555$g
- 555$h
- 555$j
- 555$k
- 555$l
- 555$m
- 555$n
- 555$o
- 555$p
- 555$q
- 555$r
- 555$s
- 555$t
- 562$b
- 562$c
- 562$d
- 562$e
- 562$f
- 562$g
- 562$h
- 562$k
- 562$l
- 562$m
- 562$n
- 562$o
- 562$p
- 562$r
- 562$s
- 562$t

Furthermore, the format of the mapping for 511 has been corrected.

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 7 Tomás Cohen Arazi 2022-10-21 14:47:04 UTC
Pushed to master for 22.11.

Nice work everyone, thanks!