Bug 31691

Summary: Remove non-existent fields from the See-from index (Elasticsearch, MARC21)
Product: Koha Reporter: Caroline Cyr La Rose <caroline.cyr-la-rose>
Component: Searching - ElasticsearchAssignee: Caroline Cyr La Rose <caroline.cyr-la-rose>
Status: CLOSED FIXED QA Contact: Kyle M Hall <kyle>
Severity: enhancement    
Priority: P5 - low CC: david, kyle, lucas
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31690
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31693
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
This enhancement removes fields from the See-from index that don't exist in MARC21. The existing fields can be found here: https://www.loc.gov/marc/authority/ad4xx.html
Version(s) released in:
22.11.00
Attachments: Bug 31691: Remove non-existent fields from the See-from index (Elasticsearch, MARC21)
Bug 31691: Remove non-existent fields from the See-from index (Elasticsearch, MARC21)
Bug 31691: Remove non-existent fields from the See-from index (Elasticsearch, MARC21)

Description Caroline Cyr La Rose 2022-10-05 15:56:48 UTC
The See-from index for MARC21 indexes fields that don't exist in the MARC21 format.

The existing fields can be found here : https://www.loc.gov/marc/authority/ad4xx.html

Here are the fields currently indexed that are not part of MARC21.

- 411$b
- 411$m
- 411$o
- 411$r
- 430$b
- 430$c
- 430$e
- 440(abcdefghjklmnopqrstvxyz) (all of them, 440 doesn't exist at all)
- 441(abcdefghklmnoprstvxyz) (all of them, 441 doesn't exist at all)
- 444(abcdefghjklmnopqrstvxyz) (all of them, 444 doesn't exist at all)
- 447$b
- 447$e
- 447$f
- 447$h
- 447$j
- 447$k
- 447$l
- 447$m
- 447$n
- 447$o
- 447$p
- 447$q
- 447$r
- 447$s
- 447$t
- 448$b
- 448$c
- 448$d
- 448$e
- 448$f
- 448$g
- 448$h
- 448$k
- 448$l
- 448$m
- 448$n
- 448$o
- 448$p
- 448$r
- 448$s
- 448$t
- 462$b
- 462$c
- 462$d
- 462$e
- 462$f
- 462$g
- 462$h
- 462$k
- 462$l
- 462$m
- 462$n
- 462$o
- 462$p
- 462$r
- 462$s
- 462$t
- 462$v
- 462$x
- 462$y
- 462$z
Comment 1 Caroline Cyr La Rose 2022-10-05 16:03:42 UTC
Created attachment 141387 [details] [review]
Bug 31691: Remove non-existent fields from the See-from index (Elasticsearch, MARC21)

This patch removes fields from the See-from index that don't exist in
MARC21.

The existing fields can be found here: https://www.loc.gov/marc/authority/ad4xx.html

The following fields are removed:

- 411$b
- 411$m
- 411$o
- 411$r
- 430$b
- 430$c
- 430$e
- 440(abcdefghjklmnopqrstvxyz) (all of them, 440 doesn't exist at all)
- 441(abcdefghklmnoprstvxyz) (all of them, 441 doesn't exist at all)
- 444(abcdefghjklmnopqrstvxyz) (all of them, 444 doesn't exist at all)
- 447$b
- 447$e
- 447$f
- 447$h
- 447$j
- 447$k
- 447$l
- 447$m
- 447$n
- 447$o
- 447$p
- 447$q
- 447$r
- 447$s
- 447$t
- 448$b
- 448$c
- 448$d
- 448$e
- 448$f
- 448$g
- 448$h
- 448$k
- 448$l
- 448$m
- 448$n
- 448$o
- 448$p
- 448$r
- 448$s
- 448$t
- 462$b
- 462$c
- 462$d
- 462$e
- 462$f
- 462$g
- 462$h
- 462$k
- 462$l
- 462$m
- 462$n
- 462$o
- 462$p
- 462$r
- 462$s
- 462$t
- 462$v
- 462$x
- 462$y
- 462$z
Comment 2 David Nind 2022-10-10 18:57:04 UTC
Created attachment 141582 [details] [review]
Bug 31691: Remove non-existent fields from the See-from index (Elasticsearch, MARC21)

This patch removes fields from the See-from index that don't exist in
MARC21.

The existing fields can be found here: https://www.loc.gov/marc/authority/ad4xx.html

The following fields are removed:

- 411$b
- 411$m
- 411$o
- 411$r
- 430$b
- 430$c
- 430$e
- 440(abcdefghjklmnopqrstvxyz) (all of them, 440 doesn't exist at all)
- 441(abcdefghklmnoprstvxyz) (all of them, 441 doesn't exist at all)
- 444(abcdefghjklmnopqrstvxyz) (all of them, 444 doesn't exist at all)
- 447$b
- 447$e
- 447$f
- 447$h
- 447$j
- 447$k
- 447$l
- 447$m
- 447$n
- 447$o
- 447$p
- 447$q
- 447$r
- 447$s
- 447$t
- 448$b
- 448$c
- 448$d
- 448$e
- 448$f
- 448$g
- 448$h
- 448$k
- 448$l
- 448$m
- 448$n
- 448$o
- 448$p
- 448$r
- 448$s
- 448$t
- 462$b
- 462$c
- 462$d
- 462$e
- 462$f
- 462$g
- 462$h
- 462$k
- 462$l
- 462$m
- 462$n
- 462$o
- 462$p
- 462$r
- 462$s
- 462$t
- 462$v
- 462$x
- 462$y
- 462$z

Signed-off-by: David Nind <david@davidnind.com>
Comment 3 David Nind 2022-10-10 18:59:19 UTC
"Tested" by comparing the list of fields removed with the Library of Congress definitions.
Comment 4 Kyle M Hall 2022-10-21 11:44:26 UTC
Created attachment 142338 [details] [review]
Bug 31691: Remove non-existent fields from the See-from index (Elasticsearch, MARC21)

This patch removes fields from the See-from index that don't exist in
MARC21.

The existing fields can be found here: https://www.loc.gov/marc/authority/ad4xx.html

The following fields are removed:

- 411$b
- 411$m
- 411$o
- 411$r
- 430$b
- 430$c
- 430$e
- 440(abcdefghjklmnopqrstvxyz) (all of them, 440 doesn't exist at all)
- 441(abcdefghklmnoprstvxyz) (all of them, 441 doesn't exist at all)
- 444(abcdefghjklmnopqrstvxyz) (all of them, 444 doesn't exist at all)
- 447$b
- 447$e
- 447$f
- 447$h
- 447$j
- 447$k
- 447$l
- 447$m
- 447$n
- 447$o
- 447$p
- 447$q
- 447$r
- 447$s
- 447$t
- 448$b
- 448$c
- 448$d
- 448$e
- 448$f
- 448$g
- 448$h
- 448$k
- 448$l
- 448$m
- 448$n
- 448$o
- 448$p
- 448$r
- 448$s
- 448$t
- 462$b
- 462$c
- 462$d
- 462$e
- 462$f
- 462$g
- 462$h
- 462$k
- 462$l
- 462$m
- 462$n
- 462$o
- 462$p
- 462$r
- 462$s
- 462$t
- 462$v
- 462$x
- 462$y
- 462$z

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 5 Tomás Cohen Arazi 2022-10-21 14:46:59 UTC
Pushed to master for 22.11.

Nice work everyone, thanks!