Caught by unit tests added in Bug 38646, but it's not a regression of that bug. It seems that the list of languages from getLanguages is not ordered in any way, instead it relies on the implementation detail of the database server in terms of the order it returns things in. Indeed even before that bug was merged, in Koha 24.11 and 24.05, one can observe in OPAC -> Advanced search -> Language, that the order of languages is indeed a bit wrong (for example: amh, grc, ara -- Old Greek (grk) has nothing starting with "a", so it has no business being this high in the list). The solution in principle is simple, just order the returned language list (TBD what to order it by: by language code, by English name, by effective final display name????). It's also TBD whether getLanguages should sort it itself, or whether places that display languages, such as `/cgi-bin/koha/opac-search.pl`, should actually be doing the ordering in preferred fashion. In any case, the tests at t/db_dependent/Languages.t fail due to wrong order after some MariaDB update, so current order is undefined behavior. Also if we select to change the behavior of getLanguages, then it'll be trickier to backport it, since dependent Bug 38646 is enhancement due to slight change of behavior. But the backporter could easily adjust the SQL query there to make the lines "THEN CONCAT(" be selected with removing the conditional, so that the language list is always appended in parentheses (old behavior).
Also wasn't sure whether to mark it as OPAC or I18N category. Technically the user-facing display bug is in OPAC, even though also unit test regressed after MariaDB update due to the UB. Welp.
I was going to say that sorting the @languages_list at the end of getLanguages() would be the easiest solution, but that wouldn't be translation friendly... ...but it looks like we just use the description straight from the database, so that's fine. I don't recall if sort() can handle UTF-8 or not. I think it should. But if not it might be necessary to use something like Unicode::Collate's sort()
Or do we sort by code? Using Unicode::Collate might be useful with the description because that would use whatever the default is used in the Unicode Collation Algorithms... I wonder if Greek speakers would expect to find Greek at the end of the list or in the Es... This could be another case where Unicode::Collate::Locale could be useful although it runs into issues of installed locales... see bug 36947
Now after the follow-up to Bug 38646, the order of languages returned from C4::Languages::getLanguages is sorted by language code ascending. I think that's a good call, since it makes the output of the function stable, regardless of the session context (and the user language selected therein). So as far as undefined behavior went, that bug is solved. However, I keep it open and change status to "In Discussion", unless someone decides to close, because there might still be some merit to considering language order in places where users see it. I think for places where actual users sees a list of languages, they might really expect to see them ordered alphabetically by how they're displayed, but for that we might want to handle this directly at the places that display such lists in/for UI and not at getLanguages func level. That might be the most stable approach. Aforementioned places is mostly the advanced search form page, I think.
I kept the bug name
Whooo this is bad for users. There are so much languages. For example spa/Spanish is "Espagnol" in french it ends up in end of list.
Ahhh but I've tested this was already in Koha 23.11 with some misplaced languages.
*** Bug 28245 has been marked as a duplicate of this bug. ***
(In reply to Fridolin Somers from comment #7) > Ahhh but I've tested this was already in Koha 23.11 with some misplaced > languages. Yeah I think I originally reported on bug 28245 back in 2021. I think it's been a problem for a long time. Or at least the languages haven't been sorted by display name
From memory, I think sorting the languages has some challenges when it comes to CJK languages? I'm curious how other scripts like Arabic and Devanagari would sort as well. I think CJK sort by byte order which isn't great. Might be the same for other non-roman alphabet languages. But... I suppose there's nothing to do but try...
Created attachment 191748 [details] [review] Bug 39001: Sort language search limits by description Patch from commit d09c0b6
Here is a proposed simple fix using sort by description. Looks way better. I'll add a test plan if validated ;)
This a bit like for item types : https://git.koha-community.org/Koha-community/Koha/src/commit/f21255ca54ac4c774eeb8f4bbd7597ec47edde87/opac/opac-search.pl#L260
Glancing at this again, I think it could make sense to instead add a third parameter to getLanguages in C4/Languages.pm to deduplicate the sorting code and ensure the translated order is consistent in different places too? Maybe under `my $isFiltered = shift;` add something like `my $isSortedByDescription = shift;`? And then sort it in an "if" right before returning (keep the default ORDER BY in the SQL query above it). And it'd be opt-in, so it'd still default to language codes sorting as expected without explicitly wanting it sorted like for displaying in HTML template. And then as David suggested, I think the actual sorting should then use Unicode::Collate's sorting, if my memory serves me right I don't think any other ordering would be proper or sufficient. With its default config, it should be a simple one-liner just as well. And since the ES facets seem to use Unicode::Collate now too, I don't see a reason not to use it here as well. > This could be another case where Unicode::Collate::Locale could be useful although it runs into issues of installed locales... see bug 36947 I think we can ignore that, there are many places in Koha now where localization will break without these locales installed, and it's now mentioned in the wiki install guide (not sure if prominently enough, but yeah...).
(In reply to Michał from comment #14) > And then as David suggested, I think the actual sorting should then use > Unicode::Collate's sorting, if my memory serves me right I don't think any > other ordering would be proper or sufficient. With its default config, it > should be a simple one-liner just as well. And since the ES facets seem to > use Unicode::Collate now too, I don't see a reason not to use it here as > well. I think we reverted it out of the ES facets as it was causing performance problems when large numbers of facets were used :(
> I think it could make sense to instead add a third parameter to getLanguages in C4/Languages.pm +1