Bug 39001 - Language list is displayed in wrong order (getLanguages sorts it by language code; places that display language list to the end-user in UI should re-sort that by localized display name order)
Summary: Language list is displayed in wrong order (getLanguages sorts it by language ...
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: I18N/L10N (show other bugs)
Version: Main
Hardware: All All
: P5 - low trivial
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
: 28245 (view as bug list)
Depends on: 38646
Blocks:
  Show dependency treegraph
 
Reported: 2025-01-30 10:12 UTC by Michał
Modified: 2026-01-22 08:42 UTC (History)
5 users (show)

See Also:
GIT URL:
Initiative type: ---
Sponsorship status: ---
Comma delimited list of Sponsors:
Crowdfunding goal: 0
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
Bug 39001: Sort language search limits by description (1.66 KB, patch)
2026-01-21 10:13 UTC, Fridolin Somers
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Michał 2025-01-30 10:12:16 UTC
Caught by unit tests added in Bug 38646, but it's not a regression of that bug. It seems that the list of languages from getLanguages is not ordered in any way, instead it relies on the implementation detail of the database server in terms of the order it returns things in. Indeed even before that bug was merged, in Koha 24.11 and 24.05, one can observe in OPAC -> Advanced search -> Language, that the order of languages is indeed a bit wrong (for example: amh, grc, ara -- Old Greek (grk) has nothing starting with "a", so it has no business being this high in the list).

The solution in principle is simple, just order the returned language list (TBD what to order it by: by language code, by English name, by effective final display name????). It's also TBD whether getLanguages should sort it itself, or whether places that display languages, such as `/cgi-bin/koha/opac-search.pl`, should actually be doing the ordering in preferred fashion.

In any case, the tests at t/db_dependent/Languages.t fail due to wrong order after some MariaDB update, so current order is undefined behavior.

Also if we select to change the behavior of getLanguages, then it'll be trickier to backport it, since dependent Bug 38646 is enhancement due to slight change of behavior. But the backporter could easily adjust the SQL query there to make the lines "THEN CONCAT(" be selected with removing the conditional, so that the language list is always appended in parentheses (old behavior).
Comment 1 Michał 2025-01-30 10:14:19 UTC
Also wasn't sure whether to mark it as OPAC or I18N category. Technically the user-facing display bug is in OPAC, even though also unit test regressed after MariaDB update due to the UB. Welp.
Comment 2 David Cook 2025-01-30 22:29:44 UTC
I was going to say that sorting the @languages_list at the end of getLanguages() would be the easiest solution, but that wouldn't be translation friendly...

...but it looks like we just use the description straight from the database, so that's fine. 

I don't recall if sort() can handle UTF-8 or not. I think it should. But if not it might be necessary to use something like Unicode::Collate's sort()
Comment 3 David Cook 2025-01-30 23:01:19 UTC
Or do we sort by code? 

Using Unicode::Collate might be useful with the description because that would use whatever the default is used in the Unicode Collation Algorithms...

I wonder if Greek speakers would expect to find Greek at the end of the list or in the Es... 

This could be another case where Unicode::Collate::Locale could be useful although it runs into issues of installed locales... see bug 36947
Comment 4 Michał 2025-04-25 16:01:39 UTC
Now after the follow-up to Bug 38646, the order of languages returned from C4::Languages::getLanguages is sorted by language code ascending. I think that's a good call, since it makes the output of the function stable, regardless of the session context (and the user language selected therein).

So as far as undefined behavior went, that bug is solved.

However, I keep it open and change status to "In Discussion", unless someone decides to close, because there might still be some merit to considering language order in places where users see it.

I think for places where actual users sees a list of languages, they might really expect to see them ordered alphabetically by how they're displayed, but for that we might want to handle this directly at the places that display such lists in/for UI and not at getLanguages func level. That might be the most stable approach.

Aforementioned places is mostly the advanced search form page, I think.
Comment 5 Michał 2025-04-25 16:07:09 UTC
I kept the bug name
Comment 6 Fridolin Somers 2026-01-20 10:11:07 UTC
Whooo this is bad for users.
There are so much languages.
For example spa/Spanish is "Espagnol" in french it ends up in end of list.
Comment 7 Fridolin Somers 2026-01-20 10:19:25 UTC
Ahhh but I've tested this was already in Koha 23.11 with some misplaced languages.
Comment 8 David Cook 2026-01-20 22:24:41 UTC
*** Bug 28245 has been marked as a duplicate of this bug. ***
Comment 9 David Cook 2026-01-20 22:26:14 UTC
(In reply to Fridolin Somers from comment #7)
> Ahhh but I've tested this was already in Koha 23.11 with some misplaced
> languages.

Yeah I think I originally reported on bug 28245 back in 2021. I think it's been a problem for a long time.

Or at least the languages haven't been sorted by display name
Comment 10 David Cook 2026-01-20 23:36:56 UTC
From memory, I think sorting the languages has some challenges when it comes to CJK languages? 

I'm curious how other scripts like Arabic and Devanagari would sort as well. I think CJK sort by byte order which isn't great. Might be the same for other non-roman alphabet languages.

But... I suppose there's nothing to do but try...
Comment 11 Fridolin Somers 2026-01-21 10:13:06 UTC
Created attachment 191748 [details] [review]
Bug 39001: Sort language search limits by description

Patch from commit d09c0b6
Comment 12 Fridolin Somers 2026-01-21 10:14:05 UTC
Here is a proposed simple fix using sort by description.

Looks way better.

I'll add a test plan if validated ;)
Comment 14 Michał 2026-01-21 18:06:34 UTC
Glancing at this again, I think it could make sense to instead add a third parameter to getLanguages in C4/Languages.pm to deduplicate the sorting code and ensure the translated order is consistent in different places too?

Maybe under `my $isFiltered = shift;` add something like `my $isSortedByDescription = shift;`? And then sort it in an "if" right before returning (keep the default ORDER BY in the SQL query above it). And it'd be opt-in, so it'd still default to language codes sorting as expected without explicitly wanting it sorted like for displaying in HTML template.

And then as David suggested, I think the actual sorting should then use Unicode::Collate's sorting, if my memory serves me right I don't think any other ordering would be proper or sufficient. With its default config, it should be a simple one-liner just as well. And since the ES facets seem to use Unicode::Collate now too, I don't see a reason not to use it here as well.

> This could be another case where Unicode::Collate::Locale could be useful although it runs into issues of installed locales... see bug 36947

I think we can ignore that, there are many places in Koha now where localization will break without these locales installed, and it's now mentioned in the wiki install guide (not sure if prominently enough, but yeah...).
Comment 15 David Cook 2026-01-21 22:04:39 UTC
(In reply to Michał from comment #14)
> And then as David suggested, I think the actual sorting should then use
> Unicode::Collate's sorting, if my memory serves me right I don't think any
> other ordering would be proper or sufficient. With its default config, it
> should be a simple one-liner just as well. And since the ES facets seem to
> use Unicode::Collate now too, I don't see a reason not to use it here as
> well.

I think we reverted it out of the ES facets as it was causing performance problems when large numbers of facets were used :(
Comment 16 Fridolin Somers 2026-01-22 08:42:11 UTC
> I think it could make sense to instead add a third parameter to getLanguages in C4/Languages.pm
+1