Bug 39147 - Add more missing languages
Summary: Add more missing languages
Status: Pushed to main
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement
Assignee: Janusz Kaczmarek
QA Contact: Thomas Klausner
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-02-17 12:56 UTC by Janusz Kaczmarek
Modified: 2025-05-16 15:32 UTC (History)
1 user (show)

See Also:
GIT URL:
Change sponsored?: Sponsored
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
25.05.00
Circulation function:


Attachments
Bug 39147: Add more missing languages (273.09 KB, patch)
2025-02-17 14:29 UTC, Janusz Kaczmarek
Details | Diff | Splinter Review
Bug 39147: Add more missing languages (274.17 KB, patch)
2025-02-17 15:21 UTC, Janusz Kaczmarek
Details | Diff | Splinter Review
Bug 39147: Add more missing languages (274.17 KB, patch)
2025-02-17 15:44 UTC, Janusz Kaczmarek
Details | Diff | Splinter Review
Bug 39147: Add more missing languages (274.22 KB, patch)
2025-02-24 16:10 UTC, Roman Dolny
Details | Diff | Splinter Review
Bug 39147: Add more missing languages (274.27 KB, patch)
2025-03-31 09:52 UTC, Thomas Klausner
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Janusz Kaczmarek 2025-02-17 12:56:35 UTC
When languages come into facets in ES, Koha should know the names for the codes used records.  Otherwise the three-letters language code is being displayed in the facet instead of name.  Koha now (as for February 2025) knows about 180 languages, many are still missing. On the official MARC 21 language codes list (https://loc.gov/marc/languages/language_code.html) there are 485 languages. We are going to add now next 230 most popular ones (= existing in our real catalogue). 

A patch will follow.
Comment 1 Janusz Kaczmarek 2025-02-17 14:29:45 UTC
Created attachment 178140 [details] [review]
Bug 39147: Add more missing languages

When languages come into facets in ES, Koha should know the names
for the codes used records.  Otherwise the three-letters language
code is being displayed in the facet instead of name.  Koha now
(as for February 2025) knows about 180 languages, many are still
missing. On the official MARC 21 language codes list
(https://loc.gov/marc/languages/language_code.html) there are
485 languages. We are going to add now next 230 most popular
ones (= existing in our real catalogue), i.e.:
abk, ach, ada, ady, afa, ain, aka, ale, alg, alt, anp, apa, arg, arn,
art, arw, asm, ath, aus, ava, awa, aym, bad, bak, bal, bam, bas,
bat, ber, bho, bin, bre, bua, bug, cai, car, cau, ceb, cha, chb,
chg, chk, chm, chn, chr, chv, chy, cmc, cnr, cos, cpe, cpf, cpp,
crh, crp, cus, dak, dar, day, del, din, div, dra, dsb, dua, dyu,
dzo, efi, elx, ewe, ewo, fan, fat, fij, fil, fiu, fon, fry, ful,
gaa, gil, glv, grb, grn, gsw, guj, hat, haw, him, hmn, hmo, ibo,
ido, ijo, ile, ina, ine, inh, iro, jav, jpr, kaa, kab, kam, kar,
kau, kaw, kbd, khi, kik, kir, kok, kom, kon, krc, kum, lah, lam,
lim, lol, ltz, lua, lug, lui, lun, luo, lus, mad, mai, man, map,
mas, mdf, mdr, men, mkh, mnc, mos, mun, mus, mwl, myv, nai, nap,
nau, nav, nde, new, nia, nic, niu, nog, non, nso, nub, nyn, oci,
oji, ori, orm, oss, oto, paa, pap, phi, phn, pon, raj, rap, roh,
run, rup, sag, sah, sai, sal, sas, sat, scn, sco, sel, sga, sgn,
shn, sin, sio, sit, smo, sna, snk, sog, son, srd, ssa, sun, tah,
tai, tel, tgk, tig, tiv, tkl, tli, tmh, ton, tpi, tsi, tsn, tso,
tum, tup, twi, tyv, udm, uig, umb, vai, ven, vol, wak, was, wln,
wol, xal, xho, yap, ypk, zap, znd, zul

and additionally ca. 15 Polish translations to existing languages.

Subtags and names for language_subtag_registry are taken from
https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry,
English names for language_descriptions.description - from
https://www.loc.gov/marc/languages/language_code.html, and Polish names -
from https://przepisy.bn.org.pl/aneksy#25-kody-jezykow---marc-21.

Test plan:
==========
1. Hava a ES installation with reasonable number of records,
   with a variety of languages in 008/35-37 not present in
   Koha (you can modify your record with codes from the list above).
   Set FacetMaxCount to a high number (like 500 or so,
   dependiing on the size of your data set).
2. Do a search, like '*'.
3. Controll the content of the Language faset -- you should
   see some bare codes instead of language names.
4. Apply the patch -- update the database.
5. Perform the same search.
6. You should see the names in place of the codes.
7. Optionally, you can activate the Polish translation and
   you should be able to see Polish names for the added languages.

Sponsored-by: Ignatianum University in Cracow
Comment 2 Janusz Kaczmarek 2025-02-17 15:21:36 UTC
Created attachment 178141 [details] [review]
Bug 39147: Add more missing languages

When languages come into facets in ES, Koha should know the names
for the codes used records.  Otherwise the three-letters language
code is being displayed in the facet instead of name.  Koha now
(as for February 2025) knows about 180 languages, many are still
missing. On the official MARC 21 language codes list
(https://loc.gov/marc/languages/language_code.html) there are
485 languages. We are going to add now next 230 most popular
ones (= existing in our real catalogue), i.e.:
abk, ach, ada, ady, afa, ain, aka, ale, alg, alt, anp, apa, arg, arn,
art, arw, asm, ath, aus, ava, awa, aym, bad, bak, bal, bam, bas,
bat, ber, bho, bin, bre, bua, bug, cai, car, cau, ceb, cha, chb,
chg, chk, chm, chn, chr, chv, chy, cmc, cnr, cos, cpe, cpf, cpp,
crh, crp, cus, dak, dar, day, del, din, div, dra, dsb, dua, dyu,
dzo, efi, elx, ewe, ewo, fan, fat, fij, fil, fiu, fon, fry, ful,
gaa, gil, glv, grb, grn, gsw, guj, hat, haw, him, hmn, hmo, ibo,
ido, ijo, ile, ina, ine, inh, iro, jav, jpr, kaa, kab, kam, kar,
kau, kaw, kbd, khi, kik, kir, kok, kom, kon, krc, kum, lah, lam,
lim, lol, ltz, lua, lug, lui, lun, luo, lus, mad, mai, man, map,
mas, mdf, mdr, men, mkh, mnc, mos, mun, mus, mwl, myv, nai, nap,
nau, nav, nde, new, nia, nic, niu, nog, non, nso, nub, nyn, oci,
oji, ori, orm, oss, oto, paa, pap, phi, phn, pon, raj, rap, roh,
run, rup, sag, sah, sai, sal, sas, sat, scn, sco, sel, sga, sgn,
shn, sin, sio, sit, smo, sna, snk, sog, son, srd, ssa, sun, tah,
tai, tel, tgk, tig, tiv, tkl, tli, tmh, ton, tpi, tsi, tsn, tso,
tum, tup, twi, tyv, udm, uig, umb, vai, ven, vol, wak, was, wln,
wol, xal, xho, yap, ypk, zap, znd, zul

and additionally ca. 15 Polish translations to existing languages.

Subtags and names for language_subtag_registry are taken from
https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry,
English names for language_descriptions.description - from
https://www.loc.gov/marc/languages/language_code.html, and Polish names -
from https://przepisy.bn.org.pl/aneksy#25-kody-jezykow---marc-21.

Test plan:
==========
1. Hava a ES installation with reasonable number of records,
   with a variety of languages in 008/35-37 not present in
   Koha (you can modify your record with codes from the list above).
   Set FacetMaxCount to a high number (like 500 or so,
   dependiing on the size of your data set).
2. Do a search, like '*'.
3. Controll the content of the Language faset -- you should
   see some bare codes instead of language names.
4. Apply the patch -- update the database.
5. Perform the same search.
6. You should see the names in place of the codes.
7. Optionally, you can activate the Polish translation and
   you should be able to see Polish names for the added languages.

Sponsored-by: Ignatianum University in Cracow
Comment 3 Janusz Kaczmarek 2025-02-17 15:44:24 UTC
Created attachment 178143 [details] [review]
Bug 39147: Add more missing languages

When languages come into facets in ES, Koha should know the names
for the codes used records.  Otherwise the three-letters language
code is being displayed in the facet instead of name.  Koha now
(as for February 2025) knows about 180 languages, many are still
missing. On the official MARC 21 language codes list
(https://loc.gov/marc/languages/language_code.html) there are
485 languages. We are going to add now next 230 most popular
ones (= existing in our real catalogue), i.e.:
abk, ach, ada, ady, afa, ain, aka, ale, alg, alt, anp, apa, arg, arn,
art, arw, asm, ath, aus, ava, awa, aym, bad, bak, bal, bam, bas,
bat, ber, bho, bin, bre, bua, bug, cai, car, cau, ceb, cha, chb,
chg, chk, chm, chn, chr, chv, chy, cmc, cnr, cos, cpe, cpf, cpp,
crh, crp, cus, dak, dar, day, del, din, div, dra, dsb, dua, dyu,
dzo, efi, elx, ewe, ewo, fan, fat, fij, fil, fiu, fon, fry, ful,
gaa, gil, glv, grb, grn, gsw, guj, hat, haw, him, hmn, hmo, ibo,
ido, ijo, ile, ina, ine, inh, iro, jav, jpr, kaa, kab, kam, kar,
kau, kaw, kbd, khi, kik, kir, kok, kom, kon, krc, kum, lah, lam,
lim, lol, ltz, lua, lug, lui, lun, luo, lus, mad, mai, man, map,
mas, mdf, mdr, men, mkh, mnc, mos, mun, mus, mwl, myv, nai, nap,
nau, nav, nde, new, nia, nic, niu, nog, non, nso, nub, nyn, oci,
oji, ori, orm, oss, oto, paa, pap, phi, phn, pon, raj, rap, roh,
run, rup, sag, sah, sai, sal, sas, sat, scn, sco, sel, sga, sgn,
shn, sin, sio, sit, smo, sna, snk, sog, son, srd, ssa, sun, tah,
tai, tel, tgk, tig, tiv, tkl, tli, tmh, ton, tpi, tsi, tsn, tso,
tum, tup, twi, tyv, udm, uig, umb, vai, ven, vol, wak, was, wln,
wol, xal, xho, yap, ypk, zap, znd, zul

and additionally ca. 15 Polish translations to existing languages.

Subtags and names for language_subtag_registry are taken from
https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry,
English names for language_descriptions.description - from
https://www.loc.gov/marc/languages/language_code.html, and Polish names -
from https://przepisy.bn.org.pl/aneksy#25-kody-jezykow---marc-21.

Test plan:
==========
1. Hava a ES installation with reasonable number of records,
   with a variety of languages in 008/35-37 not present in
   Koha (you can modify your record with codes from the list above).
   Set FacetMaxCount to a high number (like 500 or so,
   dependiing on the size of your data set).
2. Do a search, like '*'.
3. Controll the content of the Language faset -- you should
   see some bare codes instead of language names.
4. Apply the patch -- update the database.
5. Perform the same search.
6. You should see the names in place of the codes.
7. Optionally, you can activate the Polish translation and
   you should be able to see Polish names for the added languages.

Sponsored-by: Ignatianum University in Cracow
Comment 4 Roman Dolny 2025-02-24 16:10:05 UTC
Created attachment 178581 [details] [review]
Bug 39147: Add more missing languages

When languages come into facets in ES, Koha should know the names
for the codes used records.  Otherwise the three-letters language
code is being displayed in the facet instead of name.  Koha now
(as for February 2025) knows about 180 languages, many are still
missing. On the official MARC 21 language codes list
(https://loc.gov/marc/languages/language_code.html) there are
485 languages. We are going to add now next 230 most popular
ones (= existing in our real catalogue), i.e.:
abk, ach, ada, ady, afa, ain, aka, ale, alg, alt, anp, apa, arg, arn,
art, arw, asm, ath, aus, ava, awa, aym, bad, bak, bal, bam, bas,
bat, ber, bho, bin, bre, bua, bug, cai, car, cau, ceb, cha, chb,
chg, chk, chm, chn, chr, chv, chy, cmc, cnr, cos, cpe, cpf, cpp,
crh, crp, cus, dak, dar, day, del, din, div, dra, dsb, dua, dyu,
dzo, efi, elx, ewe, ewo, fan, fat, fij, fil, fiu, fon, fry, ful,
gaa, gil, glv, grb, grn, gsw, guj, hat, haw, him, hmn, hmo, ibo,
ido, ijo, ile, ina, ine, inh, iro, jav, jpr, kaa, kab, kam, kar,
kau, kaw, kbd, khi, kik, kir, kok, kom, kon, krc, kum, lah, lam,
lim, lol, ltz, lua, lug, lui, lun, luo, lus, mad, mai, man, map,
mas, mdf, mdr, men, mkh, mnc, mos, mun, mus, mwl, myv, nai, nap,
nau, nav, nde, new, nia, nic, niu, nog, non, nso, nub, nyn, oci,
oji, ori, orm, oss, oto, paa, pap, phi, phn, pon, raj, rap, roh,
run, rup, sag, sah, sai, sal, sas, sat, scn, sco, sel, sga, sgn,
shn, sin, sio, sit, smo, sna, snk, sog, son, srd, ssa, sun, tah,
tai, tel, tgk, tig, tiv, tkl, tli, tmh, ton, tpi, tsi, tsn, tso,
tum, tup, twi, tyv, udm, uig, umb, vai, ven, vol, wak, was, wln,
wol, xal, xho, yap, ypk, zap, znd, zul

and additionally ca. 15 Polish translations to existing languages.

Subtags and names for language_subtag_registry are taken from
https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry,
English names for language_descriptions.description - from
https://www.loc.gov/marc/languages/language_code.html, and Polish names -
from https://przepisy.bn.org.pl/aneksy#25-kody-jezykow---marc-21.

Test plan:
==========
1. Hava a ES installation with reasonable number of records,
   with a variety of languages in 008/35-37 not present in
   Koha (you can modify your record with codes from the list above).
   Set FacetMaxCount to a high number (like 500 or so,
   dependiing on the size of your data set).
2. Do a search, like '*'.
3. Controll the content of the Language faset -- you should
   see some bare codes instead of language names.
4. Apply the patch -- update the database.
5. Perform the same search.
6. You should see the names in place of the codes.
7. Optionally, you can activate the Polish translation and
   you should be able to see Polish names for the added languages.

Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Comment 5 Thomas Klausner 2025-03-31 09:52:58 UTC
Created attachment 179964 [details] [review]
Bug 39147: Add more missing languages

When languages come into facets in ES, Koha should know the names
for the codes used records.  Otherwise the three-letters language
code is being displayed in the facet instead of name.  Koha now
(as for February 2025) knows about 180 languages, many are still
missing. On the official MARC 21 language codes list
(https://loc.gov/marc/languages/language_code.html) there are
485 languages. We are going to add now next 230 most popular
ones (= existing in our real catalogue), i.e.:
abk, ach, ada, ady, afa, ain, aka, ale, alg, alt, anp, apa, arg, arn,
art, arw, asm, ath, aus, ava, awa, aym, bad, bak, bal, bam, bas,
bat, ber, bho, bin, bre, bua, bug, cai, car, cau, ceb, cha, chb,
chg, chk, chm, chn, chr, chv, chy, cmc, cnr, cos, cpe, cpf, cpp,
crh, crp, cus, dak, dar, day, del, din, div, dra, dsb, dua, dyu,
dzo, efi, elx, ewe, ewo, fan, fat, fij, fil, fiu, fon, fry, ful,
gaa, gil, glv, grb, grn, gsw, guj, hat, haw, him, hmn, hmo, ibo,
ido, ijo, ile, ina, ine, inh, iro, jav, jpr, kaa, kab, kam, kar,
kau, kaw, kbd, khi, kik, kir, kok, kom, kon, krc, kum, lah, lam,
lim, lol, ltz, lua, lug, lui, lun, luo, lus, mad, mai, man, map,
mas, mdf, mdr, men, mkh, mnc, mos, mun, mus, mwl, myv, nai, nap,
nau, nav, nde, new, nia, nic, niu, nog, non, nso, nub, nyn, oci,
oji, ori, orm, oss, oto, paa, pap, phi, phn, pon, raj, rap, roh,
run, rup, sag, sah, sai, sal, sas, sat, scn, sco, sel, sga, sgn,
shn, sin, sio, sit, smo, sna, snk, sog, son, srd, ssa, sun, tah,
tai, tel, tgk, tig, tiv, tkl, tli, tmh, ton, tpi, tsi, tsn, tso,
tum, tup, twi, tyv, udm, uig, umb, vai, ven, vol, wak, was, wln,
wol, xal, xho, yap, ypk, zap, znd, zul

and additionally ca. 15 Polish translations to existing languages.

Subtags and names for language_subtag_registry are taken from
https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry,
English names for language_descriptions.description - from
https://www.loc.gov/marc/languages/language_code.html, and Polish names -
from https://przepisy.bn.org.pl/aneksy#25-kody-jezykow---marc-21.

Test plan:
==========
1. Hava a ES installation with reasonable number of records,
   with a variety of languages in 008/35-37 not present in
   Koha (you can modify your record with codes from the list above).
   Set FacetMaxCount to a high number (like 500 or so,
   dependiing on the size of your data set).
2. Do a search, like '*'.
3. Controll the content of the Language faset -- you should
   see some bare codes instead of language names.
4. Apply the patch -- update the database.
5. Perform the same search.
6. You should see the names in place of the codes.
7. Optionally, you can activate the Polish translation and
   you should be able to see Polish names for the added languages.

Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Thomas Klausner <domm@plix.at>
Comment 6 Katrin Fischer 2025-04-23 05:55:01 UTC
The languages pull down in advanced search starts to become a bit challenging... Should we consider changing it to a select2 style one? (separate bug)
Comment 7 Katrin Fischer 2025-04-23 07:42:58 UTC
Pushed for 25.05!

Well done everyone, thank you!
Comment 8 Katrin Fischer 2025-05-16 15:32:24 UTC
Filed:
Bug 39924 - Improve language search option in advanced search