Other languages missing are Inuktitut (iku) and Inupiaq (ipk).

looks like it needs to be added onto multiple database tables, might try again tomorrow

I tested this, cant sign off but it worked for me

--as a side note it would be nice if the languages were sorted in alphabetical order to that it is easier to find the language you want..at the moment they seem to be sorted by when they were added to the database eg the ones Gus has added are the last languages on there.

Created attachment 47286 [details] [review]
[SIGNED-OFF] 15067 - Add additional languages to advanced search language search

run update database script, look for new laguages under 'Language:' in Advanced search.
Languages that should be added are:
1. Estonian
2. Latvian
3. Lithuanian
4. Inuktitut
5. Inupiaq

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Created attachment 47287 [details] [review]
[SIGNED-OFF] Bug 15067 - Follow up to fix sorting

It looks like C4::Languages:getLanguages relies on implicit database
order for languages. They appear to be sorted by subtag so this patch
adds that sorting to the database query to ensure the dropdown is
sorted.

Signed-off-by: Aleisha <aleishaamohia@hotmail.com>

I'd say this patch is not complete.

Looking at the other part of the files, we have, for instance:

  INSERT INTO language_descriptions(subtag, type, lang, description)
  VALUES ( 'fi', 'language', 'fi', 'Suomi');

  INSERT INTO language_descriptions(subtag, type, lang, description)
  VALUES ( 'fi', 'language', 'en', 'Finnish');

We need the language name to be translated in its own language

Moreover it seems that we need to add the country with the "region" type too.

Created attachment 84122 [details] [review]
Bug 15067: Added translations of languages to those 5 languages.

Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq are added with
their translations to the Advanced Search languages.

Created attachment 84124 [details] [review]
Bug 15067: Added translations of languages to those 5 languages.

Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq are added with
their translations to the Advanced Search languages.

Test Plan:
1. From homepage, go to Advanced search
2. Click on More options next to the Search button
3. Scroll down to the Limits section
4. Click on Language: No limit
5. Find Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq at the
bottom of the dropdown.
6. They should all be there.
7. Estonian and Latvian are repeated, this need to be fixed.
8. They need to be put in alphabetical order.

Created attachment 86122 [details] [review]
Bug 15067: Added translations of languages to those 5 languages.

Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq are added with
their translations to the Advanced Search languages.

Test Plan:
1. From homepage, go to Advanced search
2. Click on More options next to the Search button
3. Scroll down to the Limits section
4. Click on Language: No limit
5. Find Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq at the
bottom of the dropdown.
6. They should all be there.
7. Estonian and Latvian are repeated, this need to be fixed.
8. They need to be put in alphabetical order.

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
The five languages are present, sorted alphabetically, and are not
repeated.

Hi Hayley, 

some unwanted changes have slipped into the last patch:

 Koha/Illbackends/BLDSS                                |  1 +
 Koha/Illbackends/koha-ill-freeform                    |  1 +
 .../mysql/atomicupdate/bug_15067-add_newlanguages.sql | 19 +++++++++++++++++++
 installer/data/mysql/mandatory/subtag_registry.sql    | 15 +++++++++++++++
 koha-ill-freeform                                     |  1 +
 5 files changed, 37 insertions(+)
 create mode 160000 Koha/Illbackends/BLDSS
 create mode 160000 Koha/Illbackends/koha-ill-freeform
 create mode 160000 koha-ill-freeform

Can you please fix the patch and switch back to signed off?

Created attachment 89642 [details] [review]
[SIGNED OFF] Bug 15067: Added translations of languages to those 5 languages

Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq are added with
their translations to the Advanced Search languages.

Test Plan:
1. From homepage, go to Advanced search
2. Click on More options next to the Search button
3. Scroll down to the Limits section
4. Click on Language: No limit
5. Find Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq at the
bottom of the dropdown.
6. They should all be there.
7. Estonian and Latvian are repeated, this need to be fixed.
8. They need to be put in alphabetical order.

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
The five languages are present, sorted alphabetically, and are not
repeated.

Hi Hayley,

wouldn't make ordering by description make more sense? The codes might not reflect the translation, making the list appear unordered again.

There is also a problem found by the QA tools:

 FAIL	C4/Languages.pm
   OK	  critic
   OK	  forbidden patterns
   OK	  git manipulation
   FAIL	  pod
		*** ERROR: 
		Spurious =cut command
		 
		 in file C4/Languages.pm

Can you please check?

Created attachment 121239 [details] [review]
Bug 15067: Follow up to fix sorting

It looks like C4::Languages:getLanguages relies on implicit database
order for languages. They appear to be sorted by subtag so this patch
adds that sorting to the database query to ensure the dropdown is
sorted.

Signed-off-by: Aleisha <aleishaamohia@hotmail.com>

Created attachment 121240 [details] [review]
Bug 15067: Add additional languages to advanced search language search

run update database script, look for new laguages under 'Language:' in Advanced search.
Languages that should be added are:
1. Estonian
2. Latvian
3. Lithuanian
4. Inuktitut
5. Inupiaq

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Created attachment 121241 [details] [review]
Bug 15067: Added translations of languages to those 5 languages

Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq are added with
their translations to the Advanced Search languages.

Test Plan:
1. From homepage, go to Advanced search
2. Click on More options next to the Search button
3. Scroll down to the Limits section
4. Click on Language: No limit
5. Find Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq at the
bottom of the dropdown.
6. They should all be there.
7. Estonian and Latvian are repeated, this need to be fixed.
8. They need to be put in alphabetical order.

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
The five languages are present, sorted alphabetically, and are not
repeated.

Hi,

Sorting is now done on description.

Also, QA tools is now ok:

koha-qa.pl -c 3 -v 2

 OK	C4/Languages.pm
   OK	  critic
   OK	  forbidden patterns
   OK	  git manipulation
   OK	  pod
   OK	  pod coverage
   OK	  spelling
   OK	  valid

Created attachment 121260 [details] [review]
Bug 15067: Follow up to fix sorting

It looks like C4::Languages:getLanguages relies on implicit database
order for languages. They appear to be sorted by subtag so this patch
adds that sorting to the database query to ensure the dropdown is
sorted.

Signed-off-by: Aleisha <aleishaamohia@hotmail.com>

Signed-off-by: David Nind <david@davidnind.com>

Created attachment 121261 [details] [review]
Bug 15067: Add additional languages to advanced search language search

run update database script, look for new laguages under 'Language:' in Advanced search.
Languages that should be added are:
1. Estonian
2. Latvian
3. Lithuanian
4. Inuktitut
5. Inupiaq

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: David Nind <david@davidnind.com>

Created attachment 121262 [details] [review]
Bug 15067: Added translations of languages to those 5 languages

Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq are added with
their translations to the Advanced Search languages.

Test Plan:
1. From homepage, go to Advanced search
2. Click on More options next to the Search button
3. Scroll down to the Limits section
4. Click on Language: No limit
5. Find Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq at the
bottom of the dropdown.
6. They should all be there.
7. Estonian and Latvian are repeated, this need to be fixed.
8. They need to be put in alphabetical order.

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
The five languages are present, sorted alphabetically, and are not
repeated.

Signed-off-by: David Nind <david@davidnind.com>

Created attachment 123140 [details] [review]
Bug 15067: Follow up to fix sorting

It looks like C4::Languages:getLanguages relies on implicit database
order for languages. They appear to be sorted by subtag so this patch
adds that sorting to the database query to ensure the dropdown is
sorted.

Signed-off-by: Aleisha <aleishaamohia@hotmail.com>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Created attachment 123141 [details] [review]
Bug 15067: Add additional languages to advanced search language search

run update database script, look for new laguages under 'Language:' in Advanced search.
Languages that should be added are:
1. Estonian
2. Latvian
3. Lithuanian
4. Inuktitut
5. Inupiaq

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Created attachment 123142 [details] [review]
Bug 15067: Added translations of languages to those 5 languages

Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq are added with
their translations to the Advanced Search languages.

Test Plan:
1. From homepage, go to Advanced search
2. Click on More options next to the Search button
3. Scroll down to the Limits section
4. Click on Language: No limit
5. Find Estonian, Latvian, Lithuanian, Inuktitut and Inupiaq at the
bottom of the dropdown.
6. They should all be there.
7. Estonian and Latvian are repeated, this need to be fixed.
8. They need to be put in alphabetical order.

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
The five languages are present, sorted alphabetically, and are not
repeated.

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Created attachment 123143 [details] [review]
Bug 15067: (QA follow-up) Ensure update is idempotent

These tables really shoul;d have some unique keys, we need a test so we can add
those

This ended up being a bigger work than expected, RM feel free to reject or move
to another bug and let the risk of duplicated languages

This is an old one, let's get it in :-)

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

About the warns in the db rev, don't we actually want to explode instead?

Pushed to master for 21.11, thanks to everybody involved!

(In reply to Jonathan Druart from comment #28)
> About the warns in the db rev, don't we actually want to explode instead?

DB rev adjusted.

Created attachment 128105 [details] [review]
Bug 15067: (QA follow-up) Add missing UNIQUE constraint in kohastructure.sql

To test:
1. Run:
   $ reset_all
   $ koha-mysql kohadev
   > SHOW CREATE TABLE language_descriptions;
=> FAIL: There is KEY `subtag_type_lang` (`subtag`,`type`,`lang`)
2. Apply this patch
3. Repeat 1
=> SUCCESS: UNIQUE KEY `subtag_type_lang` (`subtag`,`type`,`lang`)

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

The latest patch should be pushed before the 21.11 packages are out. If not, a new DBREv is needed for fresh 21.11.00 installs. Upgrading works fine.

Created attachment 128112 [details] [review]
Bug 15067: (QA follow-up) Add missing UNIQUE constraint in kohastructure.sql

To test:
1. Run:
   $ reset_all
   $ koha-mysql kohadev
   > SHOW CREATE TABLE language_descriptions;
=> FAIL: There is KEY `subtag_type_lang` (`subtag`,`type`,`lang`)
2. Apply this patch
3. Repeat 1
=> SUCCESS: UNIQUE KEY `subtag_type_lang` (`subtag`,`type`,`lang`)

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: David Nind <david@davidnind.com>

(In reply to Tomás Cohen Arazi from comment #32)
> The latest patch should be pushed before the 21.11 packages are out. If not,
> a new DBREv is needed for fresh 21.11.00 installs. Upgrading works fine.

We agreed the patch can wait for .01 so I'm moving to a separate bug, to better handle it by the QA team.

Has anyone noticed bugs 29631 and 29641? They seem to be caused by the unique keys introduced here. Shouldn't this bug revert to QA needed status?

Upgrade to 21.06.00.012 [16:41:31]: Bug 15067 - Add missing languages
ERROR: {UNKNOWN}: DBI Exception: DBD::mysql::db do failed: Duplicate entry 'IN-region' for key 'uniq_lang' at /usr/share/koha/lib/C4/Installer.pm line 738

(In reply to MD NAZRUL ISLAM from comment #36)
> Upgrade to 21.06.00.012 [16:41:31]: Bug 15067 - Add missing languages
> ERROR: {UNKNOWN}: DBI Exception: DBD::mysql::db do failed: Duplicate entry
> 'IN-region' for key 'uniq_lang' at /usr/share/koha/lib/C4/Installer.pm line
> 738

See bug 29631.