Bug 17178

Summary: Add a popup/keyboard shortcuts for diacritics and symbols in the advanced cataloging editor
Product: Koha Reporter: Nick Clemens <nick>
Component: CatalogingAssignee: Agustín Moyano <agustinmoyano>
Status: CLOSED FIXED QA Contact: Josef Moravec <josef.moravec>
Severity: enhancement    
Priority: P5 - low CC: 1joynelson, agustinmoyano, fridolin.somers, irma, jonathan.druart, josef.moravec, m.de.rooy, margaret, martin.renvoize, tomascohen, wizzyrea
Version: master   
Hardware: All   
OS: All   
URL: https://gitlab.com/thekesolutions/Koha/tree/bug-17178
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
19.11.00
Bug Depends on: 21411    
Bug Blocks: 23231, 23232, 23396    
Attachments: Bug 17178: Add shortcut to keyboard_shortcut table
Bug 17178: Add shortcut to keyboard_shortcut table
Bug 17178: Add shortcut to keyboard_shortcut table
Bug 17178: Add virtual keyboard to advanced cataloguing editor
Bug 17178: Add shortcut to keyboard_shortcut table
Bug 17178: Add virtual keyboard to advanced cataloguing editor
Bug 17178: Add shortcut to keyboard_shortcut table
Bug 17178: Add virtual keyboard to advanced cataloguing editor
Bug 17178: Add shortcut to keyboard_shortcut table
Bug 17178: Add virtual keyboard to advanced cataloguing editor
Bug 17178: Add shortcut to keyboard_shortcut table
Bug 17178: Add virtual keyboard to advanced cataloguing editor
Bug 17178: (QA follow-up) Make DB update idempotent

Description Nick Clemens 2016-08-24 00:29:14 UTC
Work to be done:

1) Add a dialog, accessible by keyboard shortcut, that shows a variety of special characters that can then be entered by clicking or choosing a keyboard shortcut.
2) These characters should include, at a minimum, Spanish language characters and the copyright/published symbols.
Comment 1 Irma Birchall 2016-10-10 03:48:27 UTC
3) Also macrons like in the word ‘Māori’ should be included.
A macron is a line above a vowel to indicate that it should be spoken as a long vowel: ā, ē, ī, ō, ū, Ā, Ē, Ī, Ō and Ū.  The Māori word for macron is tohutō (or pōtae - hat). For more detailed information on its use, see Māori Orthographic Conventions.
Comment 2 Tomás Cohen Arazi 2019-04-17 14:50:07 UTC
Maybe using this?

https://www.npmjs.com/package/simple-keyboard-layouts
Comment 3 Agustín Moyano 2019-04-24 00:29:02 UTC
I see advanced editor already has this functionality.. 

Is this intended for basic editor, or is this duplicate of 18417?
Comment 4 Nick Clemens 2019-04-24 10:20:07 UTC
I think the idea here is broader than the other bug.

The idea would be a graphical interface to select accented characters when using a keyboard that does not contains those characters or a way to easily enter them.

i.e. 
I would be able to press a keyboard-shortcut
A modal or popup would open that would contain a character map, showing various forms of different letters with accents,umlauts, etc.
I would be able to use either the mouse or keys on the beyboard to select the letter I want
The letter would then be inserted where the cursor was located in the editor
Comment 5 Agustín Moyano 2019-05-15 13:37:16 UTC
Created attachment 89784 [details] [review]
Bug 17178: Add shortcut to keyboard_shortcut table

This patch adds Ctrl-K shortcut to toggle virtual keyboard shortcut

Sponsored-by: Round Rock Public Library
Comment 6 Agustín Moyano 2019-05-15 13:48:45 UTC
Sorry, last patch gives 'request entity too large'

Please use https://gitlab.com/thekesolutions/Koha/tree/bug-17178

The patches are the last 2 commits.
Comment 7 Agustín Moyano 2019-05-15 14:04:17 UTC
> Please use https://gitlab.com/thekesolutions/Koha/tree/bug-17178

I dont know if I expressed my self correctly..
I ment use this repo https://gitlab.com/thekesolutions/Koha with branch bug-17178 

Thanks
Comment 8 Liz Rea 2019-05-15 16:59:44 UTC
Hi,

For the Māori keyboard, I didn't see how to add the tohutō, we may need to add a layout for it like they recommend here? https://github.com/hodgef/simple-keyboard-layouts/wiki/Supported-Layouts

I know that it's a relatively obscure language to ensure support for, but it's important because Koha comes from New Zealand.

Otherwise, functionally this works fine. I think if we just add a layout for Māori, either just for us or to the originating project, this should be good to go.

I might be convinced to make all of that a follow up bug as well, so this one can move on. :)

Does anyone else have opinions about this?

Cheers,
Liz
Comment 9 Tomás Cohen Arazi 2019-05-15 17:37:02 UTC
(In reply to Liz Rea from comment #8)
> 
> I know that it's a relatively obscure language to ensure support for, but
> it's important because Koha comes from New Zealand.

We should support it.

> Otherwise, functionally this works fine. I think if we just add a layout for
> Māori, either just for us or to the originating project, this should be good
> to go.
> 
> I might be convinced to make all of that a follow up bug as well, so this
> one can move on. :)

This seriously belongs to a separate bug.
Comment 10 Liz Rea 2019-05-15 19:00:48 UTC
The feature works well and it will be a relatively small job to add the Māori keyboard layout, good for the academy possibly. Will file another bug to be sure we add it.

Liz
Comment 11 Liz Rea 2019-05-15 19:03:05 UTC
Created attachment 89795 [details] [review]
Bug 17178: Add shortcut to keyboard_shortcut table

This patch adds Ctrl-K shortcut to toggle virtual keyboard shortcut

Sponsored-by: Round Rock Public Library

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Comment 12 Liz Rea 2019-05-15 19:04:14 UTC
still a problem with request entity too large, but you can get the patch (which should have an implied sign off from me) from Theke's gitlab.

Liz
Comment 13 Josef Moravec 2019-05-27 09:29:29 UTC
Just one question:

why are all the language and layout files added to the codebase, when you need only the ones named all.js?
Comment 14 Agustín Moyano 2019-05-27 14:48:21 UTC
(In reply to Josef Moravec from comment #13)
> Just one question:
> 
> why are all the language and layout files added to the codebase, when you
> need only the ones named all.js?

Hi Josef, all.js is constructed by concatenating all js files.. 

So, if you want to modify one language, or add a new one you should:

1. remove all.js
2. place the new file in that directory (or modify an existing file)
3. cat *.js > all.js
Comment 15 Tomás Cohen Arazi 2019-05-31 18:50:06 UTC
(In reply to Agustín Moyano from comment #14)
> (In reply to Josef Moravec from comment #13)
> > Just one question:
> > 
> > why are all the language and layout files added to the codebase, when you
> > need only the ones named all.js?
> 
> Hi Josef, all.js is constructed by concatenating all js files.. 
> 
> So, if you want to modify one language, or add a new one you should:
> 
> 1. remove all.js
> 2. place the new file in that directory (or modify an existing file)
> 3. cat *.js > all.js

Agustin, I think Josef's question is if we really need to ship those files with the project, or we can just remove them to save space.
Comment 16 Agustín Moyano 2019-05-31 19:20:55 UTC
(In reply to Tomás Cohen Arazi from comment #15)
> (In reply to Agustín Moyano from comment #14)
> > (In reply to Josef Moravec from comment #13)
> > > Just one question:
> > > 
> > > why are all the language and layout files added to the codebase, when you
> > > need only the ones named all.js?
> > 
> > Hi Josef, all.js is constructed by concatenating all js files.. 
> > 
> > So, if you want to modify one language, or add a new one you should:
> > 
> > 1. remove all.js
> > 2. place the new file in that directory (or modify an existing file)
> > 3. cat *.js > all.js
> 
> Agustin, I think Josef's question is if we really need to ship those files
> with the project, or we can just remove them to save space.

If modifications from layout / languages are required, it would be rather inconvenient to have to import those files, make the adjustments and remove them... 

I'm not sure which is best.. 

To make modifications, surely the most convenient option is to have those files, but it's clear there are a lot of files.. 

maybe we could have a separate repo somehow and import / copy the resultant all.js
Comment 17 Josef Moravec 2019-06-03 08:17:05 UTC
(In reply to Agustín Moyano from comment #16)
> (In reply to Tomás Cohen Arazi from comment #15)
> > (In reply to Agustín Moyano from comment #14)
> > > (In reply to Josef Moravec from comment #13)
> > > > Just one question:
> > > > 
> > > > why are all the language and layout files added to the codebase, when you
> > > > need only the ones named all.js?
> > > 
> > > Hi Josef, all.js is constructed by concatenating all js files.. 
> > > 
> > > So, if you want to modify one language, or add a new one you should:
> > > 
> > > 1. remove all.js
> > > 2. place the new file in that directory (or modify an existing file)
> > > 3. cat *.js > all.js
> > 
> > Agustin, I think Josef's question is if we really need to ship those files
> > with the project, or we can just remove them to save space.
> 
> If modifications from layout / languages are required, it would be rather
> inconvenient to have to import those files, make the adjustments and remove
> them... 
> 
> I'm not sure which is best.. 
> 
> To make modifications, surely the most convenient option is to have those
> files, but it's clear there are a lot of files.. 
> 
> maybe we could have a separate repo somehow and import / copy the resultant
> all.js

But why don't to contribute back to Mottie Keyboard?

Also, these files are minified, I don't think anybody would like to edit minified files...
Comment 18 Agustín Moyano 2019-06-03 13:38:07 UTC
(In reply to Josef Moravec from comment #17)
> But why don't to contribute back to Mottie Keyboard?
> 
> Also, these files are minified, I don't think anybody would like to edit
> minified files...

Hi Josef.. all.js is not part of Mottie keyboard.. and I found no other way to include all languages, that didn't require to parse all js files with perl, just to write an array of languages in the template.

Maybe we could have a fork of that project, so we can PR on that project, and in the same time we could have 'releases' with all.js compiled and included.
Comment 19 Tomás Cohen Arazi 2019-06-24 13:36:09 UTC
(In reply to Josef Moravec from comment #17)
> > 
> > I'm not sure which is best.. 
> > 
> > To make modifications, surely the most convenient option is to have those
> > files, but it's clear there are a lot of files.. 
> > 
> > maybe we could have a separate repo somehow and import / copy the resultant
> > all.js
> 
> But why don't to contribute back to Mottie Keyboard?
> 
> Also, these files are minified, I don't think anybody would like to edit
> minified files...

Josef, Agustin removed the files from the commit and only have all.js now. We propose to keep our own community fork (in next dev meeting agenda) just in case someone wants to add any special layout that doesn't fit upstream, and we will document how all.js is generated (basically cat *.js > all.js) so we can keep it up-to-date with upstream (probably automated build of all.js).
Comment 20 Tomás Cohen Arazi 2019-06-24 19:42:08 UTC
Created attachment 90975 [details] [review]
Bug 17178: Add shortcut to keyboard_shortcut table

This patch adds Ctrl-K shortcut to toggle virtual keyboard shortcut

Sponsored-by: Round Rock Public Library
Comment 21 Tomás Cohen Arazi 2019-06-24 19:42:22 UTC
Created attachment 90976 [details] [review]
Bug 17178: Add virtual keyboard to advanced cataloguing editor

This patch adds Mottie keyboard (https://github.com/Mottie/Keyboard) to advanced editor.

To test:
1) Apply patches
2) perl install/data/mysql/updatedatabase.pl
3) Enable EnableAdvancedCatalogingEditor system preference
4) Go to cataloguing and to Advanced editor
5) Click on Keyboard shortcuts
SUCCESS => the keyboard shortcut "Ctrl-K" should be displayed, with "Toggle keyboard" as description
6) press Ctrl-K
SUCCESS => a virtual keyboard should be displayed, and a new toolbar button labeled "Keyboard layout" should appear.
	=> when you press a letter on both the physical and virtual keyboard, they should be added to the editor.
7) Click on "Keyboard layout"
SUCCESS => a modal should appear, where you can filter and select keyboard layouts.
	=> when you select a layout, it should be reflected on the keyboard.
	=> when you close the modal without selecting a layout, it should keep using the previous layout.
8) Sign off

Sponsored-by: Round Rock Public Library
Comment 22 Josef Moravec 2019-06-25 20:47:32 UTC
Created attachment 91003 [details] [review]
Bug 17178: Add shortcut to keyboard_shortcut table

This patch adds Ctrl-K shortcut to toggle virtual keyboard shortcut

Sponsored-by: Round Rock Public Library

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 23 Josef Moravec 2019-06-25 20:47:47 UTC
Created attachment 91004 [details] [review]
Bug 17178: Add virtual keyboard to advanced cataloguing editor

This patch adds Mottie keyboard (https://github.com/Mottie/Keyboard) to advanced editor.

To test:
1) Apply patches
2) perl install/data/mysql/updatedatabase.pl
3) Enable EnableAdvancedCatalogingEditor system preference
4) Go to cataloguing and to Advanced editor
5) Click on Keyboard shortcuts
SUCCESS => the keyboard shortcut "Ctrl-K" should be displayed, with "Toggle keyboard" as description
6) press Ctrl-K
SUCCESS => a virtual keyboard should be displayed, and a new toolbar button labeled "Keyboard layout" should appear.
	=> when you press a letter on both the physical and virtual keyboard, they should be added to the editor.
7) Click on "Keyboard layout"
SUCCESS => a modal should appear, where you can filter and select keyboard layouts.
	=> when you select a layout, it should be reflected on the keyboard.
	=> when you close the modal without selecting a layout, it should keep using the previous layout.
8) Sign off

Sponsored-by: Round Rock Public Library

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 24 Josef Moravec 2019-06-25 20:49:19 UTC
I am not big fan of forking, but if community will be OK with, I am OK with it too.
Comment 25 Tomás Cohen Arazi 2019-06-25 21:16:25 UTC
(In reply to Josef Moravec from comment #24)
> I am not big fan of forking, but if community will be OK with, I am OK with
> it too.

Well, there's no actual fork anyway. We were just thinking in advance about the case someone wants to add new layouts that don't match upstream requirements. So we are good.

We forgot to add Liz's SO line from when the patches were in a remote branch due to size limitations in Bugzilla.
Comment 26 Jonathan Druart 2019-06-25 23:14:44 UTC
1. ctrl-k may not be appropriate as it will put the focus in the search input of the browser (use ctrl-k twice)

2. Try: ctrl-k then <tab>
=> it won't change the focus to the next field/value
Is it expected?

3. The INSERT SQL statement must be using IGNORE
Comment 27 Jonathan Druart 2019-06-25 23:20:20 UTC
Also, maybe I read too fast the comments from this bug report, but can you explain:

Why https://github.com/Mottie/Keyboard over https://www.npmjs.com/package/simple-keyboard-layouts finally?

(In reply to Tomás Cohen Arazi from comment #25)
> Well, there's no actual fork anyway. We were just thinking in advance about
> the case someone wants to add new layouts that don't match upstream
> requirements. So we are good.

Why are we talking about forking? What are those requirements? Did we contact them?
Comment 28 Agustín Moyano 2019-06-26 01:27:27 UTC
(In reply to Jonathan Druart from comment #27)
> Also, maybe I read too fast the comments from this bug report, but can you
> explain:
> 
> Why https://github.com/Mottie/Keyboard over
> https://www.npmjs.com/package/simple-keyboard-layouts finally?
> 
> (In reply to Tomás Cohen Arazi from comment #25)
> > Well, there's no actual fork anyway. We were just thinking in advance about
> > the case someone wants to add new layouts that don't match upstream
> > requirements. So we are good.
> 
> Why are we talking about forking? What are those requirements? Did we
> contact them?

Hi Jonathan, both seem great, but I found a working example of integration with CodeMirror for Mottie.. couldn't find anyone for simple-keyboard-layouts
Comment 29 Agustín Moyano 2019-06-26 01:45:19 UTC
(In reply to Jonathan Druart from comment #27)
> 
> Why are we talking about forking? What are those requirements? Did we
> contact them?

In order to have a list of available layouts, and have a modal window to select and filter them, you have to load them in advance.

I could have had perl scan layouts directory, get all files in an array and pass it to the template to make the apropiate includes, or what I thought it was simpler.. to just "cat *.js > all.min.js" and include that single file.

If everyone thinks the first choice is best, I have absolutly no problem to chenge the code.

I'm still new to developing in Koha and Perl (coming from NodeJS universe, this still feels a little strange, hehe), so please be patient with me.
Comment 30 Agustín Moyano 2019-06-26 01:52:01 UTC
(In reply to Jonathan Druart from comment #26)
> 1. ctrl-k may not be appropriate as it will put the focus in the search
> input of the browser (use ctrl-k twice)
> 
> 2. Try: ctrl-k then <tab>
> => it won't change the focus to the next field/value
> Is it expected?
> 
> 3. The INSERT SQL statement must be using IGNORE

Thanks.. I hadn't notice.. any suggestions for a default key?

Maybe Shift-Ctrl-K?
Comment 31 Jonathan Druart 2019-06-26 13:47:06 UTC
(In reply to Agustín Moyano from comment #29)
> (In reply to Jonathan Druart from comment #27)
> > 
> > Why are we talking about forking? What are those requirements? Did we
> > contact them?
> 
> In order to have a list of available layouts, and have a modal window to
> select and filter them, you have to load them in advance.
> 
> I could have had perl scan layouts directory, get all files in an array and
> pass it to the template to make the apropiate includes, or what I thought it
> was simpler.. to just "cat *.js > all.min.js" and include that single file.
> 
> If everyone thinks the first choice is best, I have absolutly no problem to
> chenge the code.
> 
> I'm still new to developing in Koha and Perl (coming from NodeJS universe,
> this still feels a little strange, hehe), so please be patient with me.

I think I prefer an explicit list of layouts. We could even have a syspref later, to select some of them instead of all (?)
Comment 32 Martin Renvoize 2019-06-27 06:55:25 UTC
So to summarise after the community meeting we decided not to fork the upstream project, but rather encourage contributions to be upstreamed.

What we do need, however, is a well documented and regularly actioned way of keeping this dependency up to date (we have other dependencies too which would benefit from a workflow as well).

I proposed that perhaps we should have a topic expert whose responsibility it was to update such a dependency each cycle, but perhaps it could fall to the RM as part of their release process so long as we make it as streamlined and simple as possible.
Comment 33 Martin Renvoize 2019-06-28 13:50:17 UTC
We're not quite there yet with this one... setting back to NSO.

There appears to still be some conversations going on here and I'd really like to see a clear route for ongoing maintenance of this new dependancy mapped out before pushing it.

Also, I've gone for NSO as Liz's SO line is still missing and I believe the patch has changed somewhat since she tested..
Comment 34 Agustín Moyano 2019-06-28 14:33:42 UTC
Created attachment 91077 [details] [review]
Bug 17178: Add shortcut to keyboard_shortcut table

This patch adds Ctrl-K shortcut to toggle virtual keyboard shortcut

Sponsored-by: Round Rock Public Library

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 35 Agustín Moyano 2019-06-28 14:33:54 UTC
Created attachment 91078 [details] [review]
Bug 17178: Add virtual keyboard to advanced cataloguing editor

This patch adds Mottie keyboard (https://github.com/Mottie/Keyboard) to advanced editor.

To test:
1) Apply patches
2) perl install/data/mysql/updatedatabase.pl
3) Enable EnableAdvancedCatalogingEditor system preference
4) Go to cataloguing and to Advanced editor
5) Click on Keyboard shortcuts
SUCCESS => the keyboard shortcut "Ctrl-K" should be displayed, with "Toggle keyboard" as description
6) press Ctrl-K
SUCCESS => a virtual keyboard should be displayed, and a new toolbar button labeled "Keyboard layout" should appear.
	=> when you press a letter on both the physical and virtual keyboard, they should be added to the editor.
7) Click on "Keyboard layout"
SUCCESS => a modal should appear, where you can filter and select keyboard layouts.
	=> when you select a layout, it should be reflected on the keyboard.
	=> when you close the modal without selecting a layout, it should keep using the previous layout.
8) Sign off

Sponsored-by: Round Rock Public Library

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 36 Agustín Moyano 2019-06-28 14:46:45 UTC
Created attachment 91079 [details] [review]
Bug 17178: Add shortcut to keyboard_shortcut table

This patch adds Ctrl-K shortcut to toggle virtual keyboard shortcut

Sponsored-by: Round Rock Public Library

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 37 Agustín Moyano 2019-06-28 14:47:01 UTC
Created attachment 91080 [details] [review]
Bug 17178: Add virtual keyboard to advanced cataloguing editor

This patch adds Mottie keyboard (https://github.com/Mottie/Keyboard) to advanced editor.

To test:
1) Apply patches
2) perl install/data/mysql/updatedatabase.pl
3) Enable EnableAdvancedCatalogingEditor system preference
4) Go to cataloguing and to Advanced editor
5) Click on Keyboard shortcuts
SUCCESS => the keyboard shortcut "Ctrl-K" should be displayed, with "Toggle keyboard" as description
6) press Ctrl-K
SUCCESS => a virtual keyboard should be displayed, and a new toolbar button labeled "Keyboard layout" should appear.
	=> when you press a letter on both the physical and virtual keyboard, they should be added to the editor.
7) Click on "Keyboard layout"
SUCCESS => a modal should appear, where you can filter and select keyboard layouts.
	=> when you select a layout, it should be reflected on the keyboard.
	=> when you close the modal without selecting a layout, it should keep using the previous layout.
8) Sign off

Sponsored-by: Round Rock Public Library

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 38 Martin Renvoize 2019-06-28 15:36:01 UTC
Created attachment 91106 [details] [review]
Bug 17178: Add shortcut to keyboard_shortcut table

This patch adds Ctrl-K shortcut to toggle virtual keyboard shortcut

Sponsored-by: Round Rock Public Library

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 39 Martin Renvoize 2019-06-28 15:36:05 UTC
Created attachment 91107 [details] [review]
Bug 17178: Add virtual keyboard to advanced cataloguing editor

This patch adds Mottie keyboard (https://github.com/Mottie/Keyboard) to advanced editor.

To test:
1) Apply patches
2) perl install/data/mysql/updatedatabase.pl
3) Enable EnableAdvancedCatalogingEditor system preference
4) Go to cataloguing and to Advanced editor
5) Click on Keyboard shortcuts
SUCCESS => the keyboard shortcut "Ctrl-K" should be displayed, with "Toggle keyboard" as description
6) press Ctrl-K
SUCCESS => a virtual keyboard should be displayed, and a new toolbar button labeled "Keyboard layout" should appear.
	=> when you press a letter on both the physical and virtual keyboard, they should be added to the editor.
7) Click on "Keyboard layout"
SUCCESS => a modal should appear, where you can filter and select keyboard layouts.
	=> when you select a layout, it should be reflected on the keyboard.
	=> when you close the modal without selecting a layout, it should keep using the previous layout.
8) Sign off

Sponsored-by: Round Rock Public Library

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 40 Martin Renvoize 2019-06-28 15:36:09 UTC
Created attachment 91108 [details] [review]
Bug 17178: (QA follow-up) Make DB update idempotent

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 41 Martin Renvoize 2019-06-28 15:39:24 UTC
Right.. Thanks for the rapid follow correcting the shortcut.

Signing off and passing QA.
Comment 42 Martin Renvoize 2019-06-28 15:39:56 UTC
Nice work!

Pushed to master for 19.11.00
Comment 43 Fridolin Somers 2019-07-25 11:55:02 UTC
Enhancement not pushed to 19.05.x