Bug 17373

Summary: Elasticsearch - Authority mappings for UNIMARC
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: Searching - ElasticsearchAssignee: claire.hernandez <claire.hernandez>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: P5 - low CC: alex.arnaud, claire.hernandez, fpichenot, fridolin.somers, katrin.fischer, koha, martin.renvoize, nick, nicolas.legrand, severine.queune
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 12478, 20385    
Bug Blocks: 18825, 20261, 20273    
Attachments: Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1
Bug 17373 - Add mappings for unimarc authorities - Match index
Sample of 5 israeli authors in unimarc
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1
Bug 17373 - Add mappings for unimarc authorities - Match index
Bug 17373 - Add mappings for unimarc authorities - Match index
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1
Bug 17373 - Add mappings for unimarc authorities - Match index
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1
Bug 17373 - Add mappings for unimarc authorities - Match index

Description Jonathan Druart 2016-09-29 11:00:26 UTC
The file admin/searchengine/elasticsearch/mappings.yaml does not contain any authority mappings for UNIMARC.
Comment 1 Jonathan Druart 2016-09-29 11:00:40 UTC
I don't not plan to work on this.
Comment 2 Fridolin Somers 2016-10-13 09:46:14 UTC
Created attachment 56317 [details] [review]
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1

This patch starts the UNIMARC mapping for ES.
It is needed to test.
More patches will come.
Comment 3 Jonathan Druart 2016-10-19 15:42:49 UTC
Please provide a test plan.
Comment 4 Nicolas Legrand 2017-10-16 08:05:24 UTC
Hey Frido,

if you provide a test plan, we will sign your patch :). We currently are testing elasticsearch for the BULAC.
Comment 5 Alex Arnaud 2018-03-12 16:13:22 UTC
Created attachment 72719 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index
Comment 6 Nicolas Legrand 2018-03-13 14:10:21 UTC
Created attachment 72779 [details]
Sample of 5 israeli authors in unimarc

you can search for :

Oz
Shalev
Appelfeld
עוז
שלו
אפלפלד
Comment 7 Nicolas Legrand 2018-03-13 14:28:25 UTC
Created attachment 72780 [details] [review]
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1

This patch starts the UNIMARC mapping for ES.
It is needed to test.
More patches will come.

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Comment 8 Nicolas Legrand 2018-03-13 14:28:29 UTC
Created attachment 72781 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index

Test plan:

1. apply patches

2. switch  SearchEngine syspref to ElasticSearch

3. load new mapping
(admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1)

4. load some unimarc authorities like the sample attached to bug 17373

5. reindex authorities
(./misc/search_tools/rebuild_elastic_search.pl -d -a -v)

6. search entire notice for shalev or שלב

7. you should find stuff

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Comment 9 Nicolas Legrand 2018-03-13 15:37:17 UTC
Created attachment 72798 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index

Test plan:

1. apply patches

2. switch  SearchEngine syspref to ElasticSearch

3. load new mapping
(admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1)

4. load some unimarc authorities like the sample attached to bug 17373

5. reindex authorities
(./misc/search_tools/rebuild_elastic_search.pl -d -a -v)

6. search entire notice for shalev or שלו

7. you should find stuff

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Comment 10 Nick Clemens 2018-03-30 10:53:53 UTC
Created attachment 73461 [details] [review]
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1

This patch starts the UNIMARC mapping for ES.
It is needed to test.
More patches will come.

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 11 Nick Clemens 2018-03-30 10:53:57 UTC
Created attachment 73462 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index

Test plan:

1. apply patches

2. switch  SearchEngine syspref to ElasticSearch

3. load new mapping
(admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1)

4. load some unimarc authorities like the sample attached to bug 17373

5. reindex authorities
(./misc/search_tools/rebuild_elastic_search.pl -d -a -v)

6. search entire notice for shalev or שלו

7. you should find stuff

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 12 Nick Clemens 2018-03-30 10:54:46 UTC
I did not test live, however, the mappings code make sense, the file still loads, if they say it is correct for UNIMARC I believe them :-)
Comment 13 Jonathan Druart 2018-04-02 19:30:41 UTC
Please fix patches' author (name and email).
Comment 14 Alex Arnaud 2018-04-05 07:12:54 UTC
Created attachment 73674 [details] [review]
Bug 17373 - Elasticsearch - Authority mappings for UNIMARC - patch1

This patch starts the UNIMARC mapping for ES.
It is needed to test.
More patches will come.

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 15 Alex Arnaud 2018-04-05 07:13:01 UTC
Created attachment 73675 [details] [review]
Bug 17373 - Add mappings for unimarc authorities - Match index

Test plan:

1. apply patches

2. switch  SearchEngine syspref to ElasticSearch

3. load new mapping
(admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1)

4. load some unimarc authorities like the sample attached to bug 17373

5. reindex authorities
(./misc/search_tools/rebuild_elastic_search.pl -d -a -v)

6. search entire notice for shalev or שלו

7. you should find stuff

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 16 Jonathan Druart 2018-04-05 17:54:21 UTC
Pushed to master for 18.05, thanks to everybody involved!
Comment 17 Fridolin Somers 2018-04-06 06:45:18 UTC
Oups in second patch :
  marc_field: 235378abeijkmratsuwxyz
Is this correct ? a is there twice.
It should be alphabetical order no ?
Comment 18 Jonathan Druart 2018-04-06 14:42:58 UTC
(In reply to Fridolin SOMERS from comment #17)
> Oups in second patch :
>   marc_field: 235378abeijkmratsuwxyz
> Is this correct ? a is there twice.
> It should be alphabetical order no ?

Yes, please provide a follow-up patch here.
Comment 19 claire.hernandez@biblibre.com 2018-05-17 08:24:58 UTC
I pushed a follow-up on the bug 20390. I thing it will be easier to follow the changes and managing 2 bz for the same topic at the same time seems not relevant to me (but I do maybe a wrong choice?).
Comment 20 Fridolin Somers 2018-07-23 09:05:48 UTC
Pushed to 17.11.x for 17.11.08
Comment 21 Fridolin Somers 2018-07-23 09:06:33 UTC
Not pushed to 17.05.x, resolved