Bug 17373 - Elasticsearch - Authority mappings for UNIMARC
Summary: Elasticsearch - Authority mappings for UNIMARC
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: claire.hernandez@biblibre.com
QA Contact:
URL:
Keywords:
Depends on: 12478 20385
Blocks: 18825 20261 20273
  Show dependency treegraph
 
Reported: 2016-09-29 11:00 UTC by Jonathan Druart
Modified: 2019-10-14 19:58 UTC (History)
10 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1 (4.09 KB, patch)
2016-10-13 09:46 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 17373 - Add mappings for unimarc authorities - Match index (6.52 KB, patch)
2018-03-12 16:13 UTC, Alex Arnaud
Details | Diff | Splinter Review
Sample of 5 israeli authors in unimarc (6.79 KB, text/plain)
2018-03-13 14:10 UTC, Nicolas Legrand
Details
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1 (4.15 KB, patch)
2018-03-13 14:28 UTC, Nicolas Legrand
Details | Diff | Splinter Review
Bug 17373 - Add mappings for unimarc authorities - Match index (7.07 KB, patch)
2018-03-13 14:28 UTC, Nicolas Legrand
Details | Diff | Splinter Review
Bug 17373 - Add mappings for unimarc authorities - Match index (7.07 KB, patch)
2018-03-13 15:37 UTC, Nicolas Legrand
Details | Diff | Splinter Review
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1 (4.20 KB, patch)
2018-03-30 10:53 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 17373 - Add mappings for unimarc authorities - Match index (7.12 KB, patch)
2018-03-30 10:53 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1 (4.23 KB, patch)
2018-04-05 07:12 UTC, Alex Arnaud
Details | Diff | Splinter Review
Bug 17373 - Add mappings for unimarc authorities - Match index (7.13 KB, patch)
2018-04-05 07:13 UTC, Alex Arnaud
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2016-09-29 11:00:26 UTC
The file admin/searchengine/elasticsearch/mappings.yaml does not contain any authority mappings for UNIMARC.
Comment 1 Jonathan Druart 2016-09-29 11:00:40 UTC
I don't not plan to work on this.
Comment 2 Fridolin Somers 2016-10-13 09:46:14 UTC
Created attachment 56317 [details] [review]
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1

This patch starts the UNIMARC mapping for ES.
It is needed to test.
More patches will come.
Comment 3 Jonathan Druart 2016-10-19 15:42:49 UTC
Please provide a test plan.
Comment 4 Nicolas Legrand 2017-10-16 08:05:24 UTC
Hey Frido,

if you provide a test plan, we will sign your patch :). We currently are testing elasticsearch for the BULAC.
Comment 5 Alex Arnaud 2018-03-12 16:13:22 UTC
Created attachment 72719 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index
Comment 6 Nicolas Legrand 2018-03-13 14:10:21 UTC
Created attachment 72779 [details]
Sample of 5 israeli authors in unimarc

you can search for :

Oz
Shalev
Appelfeld
עוז
שלו
אפלפלד
Comment 7 Nicolas Legrand 2018-03-13 14:28:25 UTC
Created attachment 72780 [details] [review]
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1

This patch starts the UNIMARC mapping for ES.
It is needed to test.
More patches will come.

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Comment 8 Nicolas Legrand 2018-03-13 14:28:29 UTC
Created attachment 72781 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index

Test plan:

1. apply patches

2. switch  SearchEngine syspref to ElasticSearch

3. load new mapping
(admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1)

4. load some unimarc authorities like the sample attached to bug 17373

5. reindex authorities
(./misc/search_tools/rebuild_elastic_search.pl -d -a -v)

6. search entire notice for shalev or שלב

7. you should find stuff

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Comment 9 Nicolas Legrand 2018-03-13 15:37:17 UTC
Created attachment 72798 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index

Test plan:

1. apply patches

2. switch  SearchEngine syspref to ElasticSearch

3. load new mapping
(admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1)

4. load some unimarc authorities like the sample attached to bug 17373

5. reindex authorities
(./misc/search_tools/rebuild_elastic_search.pl -d -a -v)

6. search entire notice for shalev or שלו

7. you should find stuff

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Comment 10 Nick Clemens (kidclamp) 2018-03-30 10:53:53 UTC
Created attachment 73461 [details] [review]
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1

This patch starts the UNIMARC mapping for ES.
It is needed to test.
More patches will come.

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 11 Nick Clemens (kidclamp) 2018-03-30 10:53:57 UTC
Created attachment 73462 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index

Test plan:

1. apply patches

2. switch  SearchEngine syspref to ElasticSearch

3. load new mapping
(admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1)

4. load some unimarc authorities like the sample attached to bug 17373

5. reindex authorities
(./misc/search_tools/rebuild_elastic_search.pl -d -a -v)

6. search entire notice for shalev or שלו

7. you should find stuff

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 12 Nick Clemens (kidclamp) 2018-03-30 10:54:46 UTC
I did not test live, however, the mappings code make sense, the file still loads, if they say it is correct for UNIMARC I believe them :-)
Comment 13 Jonathan Druart 2018-04-02 19:30:41 UTC
Please fix patches' author (name and email).
Comment 14 Alex Arnaud 2018-04-05 07:12:54 UTC
Created attachment 73674 [details] [review]
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1

This patch starts the UNIMARC mapping for ES.
It is needed to test.
More patches will come.

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 15 Alex Arnaud 2018-04-05 07:13:01 UTC
Created attachment 73675 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index

Test plan:

1. apply patches

2. switch  SearchEngine syspref to ElasticSearch

3. load new mapping
(admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1)

4. load some unimarc authorities like the sample attached to bug 17373

5. reindex authorities
(./misc/search_tools/rebuild_elastic_search.pl -d -a -v)

6. search entire notice for shalev or שלו

7. you should find stuff

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 16 Jonathan Druart 2018-04-05 17:54:21 UTC
Pushed to master for 18.05, thanks to everybody involved!
Comment 17 Fridolin Somers 2018-04-06 06:45:18 UTC
Oups in second patch :
  marc_field: 235378abeijkmratsuwxyz
Is this correct ? a is there twice.
It should be alphabetical order no ?
Comment 18 Jonathan Druart 2018-04-06 14:42:58 UTC
(In reply to Fridolin SOMERS from comment #17)
> Oups in second patch :
>   marc_field: 235378abeijkmratsuwxyz
> Is this correct ? a is there twice.
> It should be alphabetical order no ?

Yes, please provide a follow-up patch here.
Comment 19 claire.hernandez@biblibre.com 2018-05-17 08:24:58 UTC
I pushed a follow-up on the bug 20390. I thing it will be easier to follow the changes and managing 2 bz for the same topic at the same time seems not relevant to me (but I do maybe a wrong choice?).
Comment 20 Fridolin Somers 2018-07-23 09:05:48 UTC
Pushed to 17.11.x for 17.11.08
Comment 21 Fridolin Somers 2018-07-23 09:06:33 UTC
Not pushed to 17.05.x, resolved