Bug 20390

Summary: Elasticsearch - Mappings for UNIMARC (updates existing)
Product: Koha Reporter: claire.hernandez <claire.hernandez>
Component: Searching - ElasticsearchAssignee: claire.hernandez <claire.hernandez>
Status: Patch doesn't apply --- QA Contact:
Severity: enhancement    
Priority: P5 - low CC: alex.arnaud, anne-cecile.bories, claire.hernandez, e.betemps, fridolin.somers, jean-manuel.broust, mathsabypro, nicolas.legrand, severine.queune
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26487
Change sponsored?: Sponsored Patch complexity: Large patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Bug Depends on: 19575, 20073    
Bug Blocks:    
Attachments: Bug 20390: UNIMARC biblio mappings WIP
Bug 20390: UNIMARC biblio mappings WIP
Follow-up: Bug 20390: UNIMARC biblio mappings WIP
Follow-up: Bug 20390: UNIMARC biblio mappings
Bug 20390: UNIMARC biblio mappings
Bug 20390: UNIMARC biblio mappings
Bug 20390: (follow-up) UNIMARC biblio mappings
Bug 20390: (follow-up) UNIMARC mappings
Bug 20390: (follow-up) UNIMARC biblio mappings
Bug 20390: (follow-up) UNIMARC mappings
Bug 20390: (follow-up) UNIMARC biblio mappings
only one rebased and reviewed
Unimarc mappings evolution for Elasticsearch
Mappings review
(follow-up) Unimarc mappings evolution for Elasticsearch
Unimarc mappings evolution for Elasticsearch
Bug 20390: mappings according to last IFLA updates

Description claire.hernandez@biblibre.com 2018-03-13 15:51:42 UTC

    
Comment 1 claire.hernandez@biblibre.com 2018-03-13 16:08:45 UTC Comment hidden (obsolete)
Comment 2 claire.hernandez@biblibre.com 2018-03-14 15:36:24 UTC Comment hidden (obsolete)
Comment 3 claire.hernandez@biblibre.com 2018-03-14 15:37:01 UTC Comment hidden (obsolete)
Comment 4 claire.hernandez@biblibre.com 2018-04-18 12:43:24 UTC Comment hidden (obsolete)
Comment 5 claire.hernandez@biblibre.com 2018-04-18 13:16:27 UTC
Created attachment 74429 [details] [review]
Bug 20390: UNIMARC biblio mappings

This is a unimarc default mappings for Koha Elasticsearch
   implementation by Claudie Trégouët and Gaetan Boisson. There will
   have updates because some discussions are in progress.

   To test:
   1) Delete data from search_marc_map, search_marc_to_field search_field
   2) Reset mappings with the url
   mappings.pl?op=reset&i_know_what_i_am_doing=1
   3) Rebuild the whole index
   rebuild_elastic_search.pl -d -b
   4) Have a look to the mappings interface
   5) Check if the recorded information in elasticsearch are with the
   expected mappings.
Comment 6 Nicolas Legrand 2018-05-03 09:35:40 UTC
mappings.yaml has been moved by bug 20073.
Comment 7 claire.hernandez@biblibre.com 2018-05-17 07:40:14 UTC
Created attachment 75388 [details] [review]
Bug 20390: UNIMARC biblio mappings

This is a unimarc default mappings for Koha Elasticsearch
   implementation by Claudie Trégouët and Gaetan Boisson. There will
   have updates because some discussions are in progress.

   To test:
   1) Delete data from search_marc_map, search_marc_to_field search_field
   2) Reset mappings with the url
   mappings.pl?op=reset&i_know_what_i_am_doing=1
   3) Rebuild the whole index
   rebuild_elastic_search.pl -d -b
   4) Have a look to the mappings interface
   5) Check if the recorded information in elasticsearch are with the
   expected mappings.

   Rebased by : Claire Hernandez <claire.hernandez@biblibre.com>
   2018/17/05
Comment 8 claire.hernandez@biblibre.com 2018-05-17 08:08:14 UTC
Created attachment 75389 [details] [review]
Bug 20390: (follow-up) UNIMARC biblio mappings

- Clean the title mapping for Unimarc: 464, 413 and 454 are kept.
If you need more, it is better to add it at the installation than removing a lot.

    - Change type of 995w for unimarc

Sponsored-by: BULAC (Bibliothèque universitaire des langues et civilisations)
Comment 9 claire.hernandez@biblibre.com 2018-05-17 08:22:39 UTC
Created attachment 75390 [details] [review]
Bug 20390: (follow-up) UNIMARC mappings

- Fix a typo in unimarc mappings (for authorities mapping)

See comment on Bug 17373
Comment 10 Séverine Queune 2018-05-30 09:17:32 UTC
Applied on sandbox#12 :

The sandbox you've requested is not ready.
Some problems occurred applying patches from bug 20390:
<h1>Something went wrong !</h1>Applying: Bug 20390: UNIMARC biblio mappings
Using index info to reconstruct a base tree...
A       etc/searchengine/elasticsearch/mappings.yaml
Falling back to patching base and 3-way merge...
Auto-merging admin/searchengine/elasticsearch/mappings.yaml
CONFLICT (content): Merge conflict in admin/searchengine/elasticsearch/mappings.yaml
error: Failed to merge in the changes.
Patch failed at 0001 Bug 20390: UNIMARC biblio mappings
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem run git bz apply --continue.
If you would prefer to skip this patch, instead run git bz apply --skip.
To restore the original branch and stop patching run git bz apply --abort.
Bug 20390 - Elasticsearch - Mappings for UNIMARC (updates existing)

75388 - Bug 20390: UNIMARC biblio mappings
75389 - Bug 20390: (follow-up) UNIMARC biblio mappings
75390 - Bug 20390: (follow-up) UNIMARC mappings

Apply? [(y)es, (n)o, (i)nteractive] Patch left in /tmp/Bug-20390-UNIMARC-biblio-mappings-lAznq5.patch .
Comment 11 claire.hernandez@biblibre.com 2018-06-25 14:00:15 UTC
20073 had been reverted, so I was waiting for its push, it now ok so 20390 applies now. Thanks Nick !
Comment 12 claire.hernandez@biblibre.com 2019-02-13 15:20:08 UTC Comment hidden (obsolete)
Comment 13 claire.hernandez@biblibre.com 2019-02-19 14:29:59 UTC Comment hidden (obsolete)
Comment 14 claire.hernandez@biblibre.com 2019-02-19 15:48:56 UTC Comment hidden (obsolete)
Comment 15 claire.hernandez@biblibre.com 2019-03-28 16:18:56 UTC Comment hidden (obsolete)
Comment 16 claire.hernandez@biblibre.com 2019-04-10 13:37:52 UTC
Created attachment 87720 [details] [review]
Unimarc mappings evolution for Elasticsearch
Comment 17 Fridolin Somers 2019-04-24 13:51:35 UTC
Looks like this need to be updated with Bug 19575 changes.
Like pubdate => date-of-publication
Comment 18 claire.hernandez@biblibre.com 2019-04-26 13:39:49 UTC Comment hidden (obsolete)
Comment 19 claire.hernandez@biblibre.com 2019-04-26 14:16:32 UTC
Created attachment 88925 [details] [review]
(follow-up) Unimarc mappings evolution for Elasticsearch
Comment 20 Fridolin Somers 2021-03-11 13:19:44 UTC
Needs rebase
Comment 21 Alex Arnaud 2021-04-01 12:01:05 UTC
Created attachment 119074 [details] [review]
Unimarc mappings evolution for Elasticsearch
Comment 22 Alex Arnaud 2021-04-01 12:01:50 UTC
Patches rebased and squashed
Comment 23 Alex Arnaud 2021-04-02 12:42:12 UTC
Created attachment 119120 [details] [review]
Bug 20390: mappings according to last IFLA updates
Comment 24 Fridolin Somers 2021-07-07 02:27:29 UTC
tpubdate is on 100a_/8-11, must be only 100a_/8
Comment 25 Fridolin Somers 2021-07-07 02:41:51 UTC
leader_/1-5 seems wrong, its leader_/0-4 like in MARC21
Comment 26 Fridolin Somers 2021-07-07 02:44:29 UTC
(In reply to Fridolin Somers from comment #25)
> leader_/1-5 seems wrong, its leader_/0-4 like in MARC21

For llength I mean
Comment 27 Fridolin Somers 2021-07-07 03:09:31 UTC
char-encoding is on 100a.
Its missing ranges.
In zebra it is : 26-27, 28-29 and 30-31
Comment 28 Fridolin Somers 2021-07-07 03:17:37 UTC
same for :
modified-code => 100a_/21
script-title => 100a_/34-35
Comment 29 Fridolin Somers 2021-07-07 03:46:11 UTC
ff8 fields also :

ff8-18    : 110a_/1
ff8-18-21 : 105a_/0
ff8-18-21 : 105a_/1
ff8-18-21 : 105a_/2
ff8-18-21 : 105a_/3
ff8-19    : 110a_/2
ff8-21    : 110a_/0
ff8-23    : 106a
ff8-29    : 105a_/8
ff8-29    : 110a_/7
ff8-30    : 105a_/9
ff8-31    : 105a_/10
ff8-31    : 110a_/9
Comment 30 Fridolin Somers 2022-01-15 04:35:52 UTC
Does not apply.

Is first patch obsolete ?

Applying only patch with "Bug 20390: ..." does not work eather.