Bug 22639

Summary: Elasticsearch ability export mappings and facets settings
Product: Koha Reporter: Michal Denar <black23>
Component: Searching - ElasticsearchAssignee: axel Amghar <axel.amghar>
Status: ASSIGNED --- QA Contact:
Severity: enhancement    
Priority: P5 - low CC: axel.amghar, black23, ere.maijala, fridolin.somers, josef.moravec, marjorie.barry-vila, rbit, severine.queune
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22637
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Bug Depends on: 20588, 22994, 23204    
Bug Blocks:    
Attachments: Bug 22639: Elasticsearch ability to export mappings and facets settings
Bug 22639: Elasticsearch ability to export mappings and facets settings
Bug 22639: Elasticsearch ability to export mappings and facets settings
Bug 22639: Elasticsearch ability to export mappings and facets settings
Bug 22639: Elasticsearch ability to export mappings and facets settings
Bug 22639: Elasticsearch ability to export mappings and facets settings

Description Michal Denar 2019-04-04 07:47:33 UTC
For easier maintance export/import of settings will be great feature.
Comment 1 Ere Maijala 2019-04-04 09:11:33 UTC
Import is sort of there already. You can define custom files in koha-conf.xml with the following settings:

<elasticsearch_index_config>
<elasticsearch_field_config>
<elasticsearch_index_mappings>

Whenever you reset the mappings in Administration, elasticsearch_index_mappings, if specified, is used to locate the default mappings. When you drop and recreate the index, elasticsearch_index_config and elasticsearch_field_config are used to create the new index.

Actually, the only thing missing is export of current mappings. I believe it wouldn't be needed regularly if you decided to use a custom mappings.yaml to define the settings.
Comment 2 Katrin Fischer 2019-04-04 09:32:40 UTC
I imagine maybe Michal was thinking about something that could be done form the GUI, not requiring access to the server to set/change file paths.

Ere - is this behavior below documented somewhere? (wiki or manual?)
Comment 3 Ere Maijala 2019-04-04 09:39:27 UTC
True, and I'm not saying they wouldn't be useful, but the files are the only place where the index and field configs are stored, so import/export would be sort of like file transfers.

Looks like I may have forgotten to add documentation to the wiki page, I'll fix that.
Comment 4 Katrin Fischer 2019-04-04 09:43:25 UTC
Thx!
Comment 5 Michal Denar 2019-04-04 09:47:41 UTC
My descriotion fault. I'm thinking about import/export from UI. I think that many librarians hasn't access to server filesystem.
Comment 6 Ere Maijala 2019-04-04 10:01:23 UTC
Makes sense. Worth noting is that index config and most aspects of field config are irrelevant unless you can also schedule a full reindexing.
Comment 7 Michal Denar 2019-04-04 10:09:01 UTC
Hi Ere,
I agree with you. This bug can solve it beuace give chance to reindex from UI:
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22637
Comment 8 Ere Maijala 2019-04-04 10:17:37 UTC
Katrin, wiki page updated: https://wiki.koha-community.org/wiki/Elasticsearch#Advanced_Index_Settings
Comment 9 Katrin Fischer 2019-04-04 17:06:43 UTC
(In reply to Ere Maijala from comment #8)
> Katrin, wiki page updated:
> https://wiki.koha-community.org/wiki/Elasticsearch#Advanced_Index_Settings

Awesome, thank you!
Comment 10 axel Amghar 2019-05-23 10:10:17 UTC Comment hidden (obsolete)
Comment 11 axel Amghar 2019-05-23 10:16:55 UTC
Hello,
I added a patch to download tables separately.
Do you think that a button who download all the tables would be good ?
Comment 12 Michal Denar 2019-05-23 10:25:32 UTC
Hello,
I'kk take look on this later today. I set this to Need sign-off.
Comment 13 axel Amghar 2019-05-23 10:35:49 UTC
(In reply to Michal Denar from comment #12)
> Hello,
> I'kk take look on this later today. I set this to Need sign-off.

There is only the export option for now
Comment 14 axel Amghar 2019-05-23 12:25:15 UTC
And do you want the possibility to download the facet order table too in bibliographic records ?
Comment 15 Michal Denar 2019-05-23 18:45:15 UTC
Everything works for me. Download of biblio facets order will be fine.

Now we can focuse on import.
Comment 16 Michal Denar 2019-05-23 18:47:00 UTC Comment hidden (obsolete)
Comment 17 Katrin Fischer 2019-05-27 05:05:35 UTC
If this is only for export, please change the bug title accordingly.
Comment 18 Michal Denar 2019-05-27 13:06:16 UTC
Hi Katrin,
I hope that folow-ups for import are comming. Export without import feature is useless.
Comment 19 axel Amghar 2019-05-27 13:11:19 UTC
(In reply to Michal Denar from comment #18)
> Hi Katrin,
> I hope that folow-ups for import are comming. Export without import feature
> is useless.

Hi, 
I could work on it next week :)
Comment 20 Michal Denar 2019-05-27 13:12:09 UTC
Hi,
great news :-)

Thank you
Comment 21 Katrin Fischer 2019-05-27 17:54:44 UTC
I think it would make sense to either wait or to export on a second depending bug, what do you think?
Comment 22 axel Amghar 2019-05-28 11:47:53 UTC Comment hidden (obsolete)
Comment 23 axel Amghar 2019-05-28 11:52:31 UTC
I added the dowload of facet order table
Comment 24 Michal Denar 2019-05-28 18:48:39 UTC Comment hidden (obsolete)
Comment 25 axel Amghar 2019-06-13 13:43:41 UTC
My previous patch must be redone because we can add the export button on the dataTable.
You have an example in cgi-bin/koha/circ/pendingreserves.pl
Like that the export will be similar with the others page and there will be less code
Comment 26 axel Amghar 2019-06-14 13:55:59 UTC Comment hidden (obsolete)
Comment 27 axel Amghar 2019-06-14 14:10:14 UTC
Created attachment 90602 [details] [review]
Bug 22639: Elasticsearch ability to export mappings and facets settings

With this patch you can download search fields , facets order, biblios and authorities table in search engine configuration

Test plan:
- apply the patch
- make sure you have "elasticsearch" as value for the syspref "SearchEngine"
- go to search engine configuration in the admin page
- in the search fields part, you have a button "Export"
- verify all formats are correct
- verify that you have the same table in your file than in the search fields table
- make the same test for
	: biblios table
	: authorities table
- Add some mappings, modify some fields, change the facets order
- Save your modifications
- Then dowload again
- Your new files must be modified

- You can remove some colomns to dowload only Name and label columns for example

ps: with this patch you can't download the facet order table
Comment 28 axel Amghar 2019-06-14 14:12:26 UTC Comment hidden (obsolete)
Comment 29 axel Amghar 2019-06-14 14:19:37 UTC
Hello,
I tried my best to remade the patch with DataTable (look 22588 for more info)
But i found lot of issues to fix:
   - the export for search field table aren't working good
   - when you try do dowload elements in the table that are 
<select>
  <option></option>
  ...
</select>
that shows all the options in the export table, but we want only the selected option

They might be some more issues.

Good Luck :)
Comment 30 Fridolin Somers 2020-10-27 08:01:15 UTC
Bug 23204 added Koha::SearchEngine::Elasticsearch::raw_elasticsearch_mappings()
Comment 31 Katrin Fischer 2023-09-23 15:44:21 UTC
*** Bug 22994 has been marked as a duplicate of this bug. ***