Bug 20390 - Elasticsearch - Mappings for UNIMARC (updates existing)
Summary: Elasticsearch - Mappings for UNIMARC (updates existing)
Status: Patch doesn't apply
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement with 1 vote (vote)
Assignee: claire.hernandez@biblibre.com
QA Contact:
URL:
Keywords:
Depends on: 19575 20073
Blocks:
  Show dependency treegraph
 
Reported: 2018-03-13 15:51 UTC by claire.hernandez@biblibre.com
Modified: 2022-01-15 04:35 UTC (History)
10 users (show)

See Also:
Change sponsored?: Sponsored
Patch complexity: Large patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 20390: UNIMARC biblio mappings WIP (24.08 KB, patch)
2018-03-13 16:08 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Bug 20390: UNIMARC biblio mappings WIP (24.37 KB, patch)
2018-03-14 15:36 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Follow-up: Bug 20390: UNIMARC biblio mappings WIP (1.41 KB, patch)
2018-03-14 15:37 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Follow-up: Bug 20390: UNIMARC biblio mappings (8.84 KB, patch)
2018-04-18 12:43 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Bug 20390: UNIMARC biblio mappings (27.94 KB, patch)
2018-04-18 13:16 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Bug 20390: UNIMARC biblio mappings (28.04 KB, patch)
2018-05-17 07:40 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Bug 20390: (follow-up) UNIMARC biblio mappings (5.89 KB, patch)
2018-05-17 08:08 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Bug 20390: (follow-up) UNIMARC mappings (916 bytes, patch)
2018-05-17 08:22 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Bug 20390: (follow-up) UNIMARC biblio mappings (11.14 KB, patch)
2019-02-13 15:20 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Bug 20390: (follow-up) UNIMARC mappings (923 bytes, patch)
2019-02-19 14:29 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Bug 20390: (follow-up) UNIMARC biblio mappings (7.00 KB, patch)
2019-02-19 15:48 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
only one rebased and reviewed (37.42 KB, patch)
2019-03-28 16:18 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Unimarc mappings evolution for Elasticsearch (37.67 KB, patch)
2019-04-10 13:37 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Mappings review (12.57 KB, patch)
2019-04-26 13:39 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
(follow-up) Unimarc mappings evolution for Elasticsearch (12.56 KB, patch)
2019-04-26 14:16 UTC, claire.hernandez@biblibre.com
Details | Diff | Splinter Review
Unimarc mappings evolution for Elasticsearch (44.18 KB, patch)
2021-04-01 12:01 UTC, Alex Arnaud
Details | Diff | Splinter Review
Bug 20390: mappings according to last IFLA updates (19.80 KB, patch)
2021-04-02 12:42 UTC, Alex Arnaud
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description claire.hernandez@biblibre.com 2018-03-13 15:51:42 UTC

    
Comment 1 claire.hernandez@biblibre.com 2018-03-13 16:08:45 UTC Comment hidden (obsolete)
Comment 2 claire.hernandez@biblibre.com 2018-03-14 15:36:24 UTC Comment hidden (obsolete)
Comment 3 claire.hernandez@biblibre.com 2018-03-14 15:37:01 UTC Comment hidden (obsolete)
Comment 4 claire.hernandez@biblibre.com 2018-04-18 12:43:24 UTC Comment hidden (obsolete)
Comment 5 claire.hernandez@biblibre.com 2018-04-18 13:16:27 UTC
Created attachment 74429 [details] [review]
Bug 20390: UNIMARC biblio mappings

This is a unimarc default mappings for Koha Elasticsearch
   implementation by Claudie Trégouët and Gaetan Boisson. There will
   have updates because some discussions are in progress.

   To test:
   1) Delete data from search_marc_map, search_marc_to_field search_field
   2) Reset mappings with the url
   mappings.pl?op=reset&i_know_what_i_am_doing=1
   3) Rebuild the whole index
   rebuild_elastic_search.pl -d -b
   4) Have a look to the mappings interface
   5) Check if the recorded information in elasticsearch are with the
   expected mappings.
Comment 6 Nicolas Legrand 2018-05-03 09:35:40 UTC
mappings.yaml has been moved by bug 20073.
Comment 7 claire.hernandez@biblibre.com 2018-05-17 07:40:14 UTC
Created attachment 75388 [details] [review]
Bug 20390: UNIMARC biblio mappings

This is a unimarc default mappings for Koha Elasticsearch
   implementation by Claudie Trégouët and Gaetan Boisson. There will
   have updates because some discussions are in progress.

   To test:
   1) Delete data from search_marc_map, search_marc_to_field search_field
   2) Reset mappings with the url
   mappings.pl?op=reset&i_know_what_i_am_doing=1
   3) Rebuild the whole index
   rebuild_elastic_search.pl -d -b
   4) Have a look to the mappings interface
   5) Check if the recorded information in elasticsearch are with the
   expected mappings.

   Rebased by : Claire Hernandez <claire.hernandez@biblibre.com>
   2018/17/05
Comment 8 claire.hernandez@biblibre.com 2018-05-17 08:08:14 UTC
Created attachment 75389 [details] [review]
Bug 20390: (follow-up) UNIMARC biblio mappings

- Clean the title mapping for Unimarc: 464, 413 and 454 are kept.
If you need more, it is better to add it at the installation than removing a lot.

    - Change type of 995w for unimarc

Sponsored-by: BULAC (Bibliothèque universitaire des langues et civilisations)
Comment 9 claire.hernandez@biblibre.com 2018-05-17 08:22:39 UTC
Created attachment 75390 [details] [review]
Bug 20390: (follow-up) UNIMARC mappings

- Fix a typo in unimarc mappings (for authorities mapping)

See comment on Bug 17373
Comment 10 Séverine Queune 2018-05-30 09:17:32 UTC
Applied on sandbox#12 :

The sandbox you've requested is not ready.
Some problems occurred applying patches from bug 20390:
<h1>Something went wrong !</h1>Applying: Bug 20390: UNIMARC biblio mappings
Using index info to reconstruct a base tree...
A       etc/searchengine/elasticsearch/mappings.yaml
Falling back to patching base and 3-way merge...
Auto-merging admin/searchengine/elasticsearch/mappings.yaml
CONFLICT (content): Merge conflict in admin/searchengine/elasticsearch/mappings.yaml
error: Failed to merge in the changes.
Patch failed at 0001 Bug 20390: UNIMARC biblio mappings
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem run git bz apply --continue.
If you would prefer to skip this patch, instead run git bz apply --skip.
To restore the original branch and stop patching run git bz apply --abort.
Bug 20390 - Elasticsearch - Mappings for UNIMARC (updates existing)

75388 - Bug 20390: UNIMARC biblio mappings
75389 - Bug 20390: (follow-up) UNIMARC biblio mappings
75390 - Bug 20390: (follow-up) UNIMARC mappings

Apply? [(y)es, (n)o, (i)nteractive] Patch left in /tmp/Bug-20390-UNIMARC-biblio-mappings-lAznq5.patch .
Comment 11 claire.hernandez@biblibre.com 2018-06-25 14:00:15 UTC
20073 had been reverted, so I was waiting for its push, it now ok so 20390 applies now. Thanks Nick !
Comment 12 claire.hernandez@biblibre.com 2019-02-13 15:20:08 UTC Comment hidden (obsolete)
Comment 13 claire.hernandez@biblibre.com 2019-02-19 14:29:59 UTC Comment hidden (obsolete)
Comment 14 claire.hernandez@biblibre.com 2019-02-19 15:48:56 UTC Comment hidden (obsolete)
Comment 15 claire.hernandez@biblibre.com 2019-03-28 16:18:56 UTC Comment hidden (obsolete)
Comment 16 claire.hernandez@biblibre.com 2019-04-10 13:37:52 UTC
Created attachment 87720 [details] [review]
Unimarc mappings evolution for Elasticsearch
Comment 17 Fridolin Somers 2019-04-24 13:51:35 UTC
Looks like this need to be updated with Bug 19575 changes.
Like pubdate => date-of-publication
Comment 18 claire.hernandez@biblibre.com 2019-04-26 13:39:49 UTC Comment hidden (obsolete)
Comment 19 claire.hernandez@biblibre.com 2019-04-26 14:16:32 UTC
Created attachment 88925 [details] [review]
(follow-up) Unimarc mappings evolution for Elasticsearch
Comment 20 Fridolin Somers 2021-03-11 13:19:44 UTC
Needs rebase
Comment 21 Alex Arnaud 2021-04-01 12:01:05 UTC
Created attachment 119074 [details] [review]
Unimarc mappings evolution for Elasticsearch
Comment 22 Alex Arnaud 2021-04-01 12:01:50 UTC
Patches rebased and squashed
Comment 23 Alex Arnaud 2021-04-02 12:42:12 UTC
Created attachment 119120 [details] [review]
Bug 20390: mappings according to last IFLA updates
Comment 24 Fridolin Somers 2021-07-07 02:27:29 UTC
tpubdate is on 100a_/8-11, must be only 100a_/8
Comment 25 Fridolin Somers 2021-07-07 02:41:51 UTC
leader_/1-5 seems wrong, its leader_/0-4 like in MARC21
Comment 26 Fridolin Somers 2021-07-07 02:44:29 UTC
(In reply to Fridolin Somers from comment #25)
> leader_/1-5 seems wrong, its leader_/0-4 like in MARC21

For llength I mean
Comment 27 Fridolin Somers 2021-07-07 03:09:31 UTC
char-encoding is on 100a.
Its missing ranges.
In zebra it is : 26-27, 28-29 and 30-31
Comment 28 Fridolin Somers 2021-07-07 03:17:37 UTC
same for :
modified-code => 100a_/21
script-title => 100a_/34-35
Comment 29 Fridolin Somers 2021-07-07 03:46:11 UTC
ff8 fields also :

ff8-18    : 110a_/1
ff8-18-21 : 105a_/0
ff8-18-21 : 105a_/1
ff8-18-21 : 105a_/2
ff8-18-21 : 105a_/3
ff8-19    : 110a_/2
ff8-21    : 110a_/0
ff8-23    : 106a
ff8-29    : 105a_/8
ff8-29    : 110a_/7
ff8-30    : 105a_/9
ff8-31    : 105a_/10
ff8-31    : 110a_/9
Comment 30 Fridolin Somers 2022-01-15 04:35:52 UTC
Does not apply.

Is first patch obsolete ?

Applying only patch with "Bug 20390: ..." does not work eather.