When resetting the Elasticsearch settings using mappings.yaml, it appears that the "Facetable" setting isn't being updated correctly. For instance, the default mappings.yaml says "publisher" is facetable, but after resetting the Elasticsearch from file, it says it's not facetable.
Ok in main... I've reset and it is setting in the database but the UI still says "No" for Facetable for publisher, local-classification, copydate. I re-indexed with the default conf, and I see "publisher__facet" and "copydate__facet" keys in the ES document despite Facetable saying "no" for publisher and copydate, and those facets not being defined in "Facet Order".
Ahhh if I save the mappings via the UI it sets "facet" to 0 for publisher, local-classification, and copydate. That would explain why they weren't showing up in the database when I looked previously...
Ahhh it's because "is_facetable" is based off Koha::SearchEngine::Elasticsearch::get_facet_fields which uses Koha::SearchFields->search( { facet_order => { '!=' => undef } }, { order_by => ['facet_order'] } )->as_list; Which has a certain logic to it but it's a problem.
Hmm it's not enough to have that "Yes" show up in the UI for publisher, local-classification, and copydate... there's something with the save overall that is blitzing it. Jeez...
That said... maybe it makes sense to only set "facet" if the mapping has a facet_order as well. Because if it doesn't, it's never going to show up anyway, and then we're creating indexed values that will never be used. But if that's the case maybe we should update the indexing to only index facet names that appear in Koha::SearchEngine::Elasticsearch->get_facet_fields() and to remove "facet: 1" from the default mappings.yaml for indexes that don't have a facet_order out of the box.
What should we do here? Option 1: - Change mappings.yaml to set facet: 0 for publisher, local-classification, copydate - This will mean that mappings.yaml and the UI are more in sync. That is, people won't look at mappings.yaml and wonder why the UI says "facetable: no" when the YAML says "facet: 1". That said, they could do that to their custom YAML and be confused still. Option 2a: - Change mappings.pl to show facet: 1 for fields like publisher, local-classification, copydate even though they don't have facet_order set. - This will mean these fields are indexed into "*__facet" fields even though those facets won't be shown. That seems suboptimal Option 2b: - Do option 2a PLUS update the indexing so that "*__facet" fields are only created if there is a facet_order for that facet, so we don't create indexes that aren't used Option 3: - Something else I haven't thought of?
Note a manifestation of this bug... if you have a default/vanilla Elasticsearch mappings, you can add the "publisher" facet immediately and have it work out of the box, because "publisher__facet" is already indexed out of the box. I had 2 similar Koha instances and one had a working publisher facet and one didn't and it's because one had an original ES config and one must've had a slightly modified one where the publisher facet had been blitzed by clicking "Save" for an unrelated configuration change. It was crazy-making for sure.
I recall domm mentioned last hackfest that saving the mappings to a file with `export_elasticsearch_mappings.pl` and then applying the file using the `--reset` flag for `rebuild_elasticsearch.pl` yielded inconsistent results in the configured mappings. I bet both are related. I think there was a bug for that too. Not facet-specific but related.
(In reply to Tomás Cohen Arazi (tcohen) from comment #8) > I recall domm mentioned last hackfest that saving the mappings to a file > with `export_elasticsearch_mappings.pl` and then applying the file using the > `--reset` flag for `rebuild_elasticsearch.pl` yielded inconsistent results > in the configured mappings. I bet both are related. I think there was a bug > for that too. Not facet-specific but related. I raised a bug for that too yesterday hehe: bug 41670 I don't know that it's related though. This facetable stuff specifically relates to changes made in bug 35138. I think bug 41670 is partially a code bug and partially the default file just being inaccurate. But with this one... I don't think the answer is quite straightforward.