Bug 30836

This also needs a guideline for QA people to follow so we don't fall out of sync again in the future.

http://irc.koha-community.org/koha/2022-05-24#i_2426285 for irc reference

Out of the box it does seem to work fine adding a mapping to the db directly, however, it relies on dynamic mapping being enabled:
https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-mapping.html

Otherwise, we will need to update the indexes in ES as well

We ceratinly need to define a good way to patch things in production for the regular user, leaving room so we don't break more advanced users setups.

If this is not doable, then we need to document better how to reset the indexes, or provide a tool to let them know they are out of sync.

This is a blocker for several enhancements.

I'm not sure if I can come up with a good solution. Automatically updating current mappings can create problems when the defaults have been customized.

What we might be able to do is to check, field by field, if the mappings have been customized, and apply any changes only on top of old defaults. I'm not sure if that's feasible in real life, but worth a thought. At least it should be possible to check that any mappings required by Koha are available regardless of their configuration, and add any that are missing completely. Perhaps a warning about updated mappings during upgrade would be useful as well.

I think that in many cases requiring resetting to defaults would be too destructive. Having to redo all customizations could be pretty a lot of work unless you manage them in a customized mappings.yaml or something. It would be useful to have a tool that would show the differences between current and default mappings. This would make it easier to determine any manual changes needed or useful.

All in all, I'm not sure I like having the database part of the configuration. Sure, the UI makes it easier to start changing the mappings, but if you need to do more, it's cumbersome. Also changing the settings can have consequences that you need to be aware of, and sometimes I feel it should not be that easy to change them.

Regardless of how the mappings update is managed, any changes will need to trigger a prominent notification that reindexing is required.

Sorry for this being a collection of random thoughts...

(In reply to Ere Maijala from comment #5)
> I'm not sure if I can come up with a good solution. Automatically updating
> current mappings can create problems when the defaults have been customized.

This reminds me that we have this problem in other parts of Koha as well, and we've never found a good solution. 

For example, MARC Bibliographic Frameworks. Also, "Koha to MARC mapping". 

(In reply to Ere Maijala from comment #5)
> What we might be able to do is to check, field by field, if the mappings
> have been customized, and apply any changes only on top of old defaults. I'm
> not sure if that's feasible in real life, but worth a thought. At least it
> should be possible to check that any mappings required by Koha are available
> regardless of their configuration, and add any that are missing completely.
> Perhaps a warning about updated mappings during upgrade would be useful as
> well.

Actually... I think that this is probably the way to go. 

I'm not familiar with the Elasticsearch mappings setup, but in theory couldn't you compare default values against user-generated values? And then update everything but those? 

And then maybe have some way of telling users which values are default and which values aren't in a visual way?  I imagine that might be easier for Elasticsearch mappings than it would for say "MARC Bibliographic Frameworks".

Isn't this what Bug 19707 is supposed to handle? I know it has failed QA for now though, so some fixing would be necessary, but if I remember correctly it adds sync/reset options for mappings.

Looks like Bug 19707 would indeed provide a lot of the functionality. What remains would be automatic updating of non-customized fields and prominent message during upgrade that prompts to merge any changes manually.