This patch intends to add support for jQuery.IME to Koha's staff client and OPAC. Koha's Indian users in particular have long been demanded a clean method for multi-lingual input support from with the browser. Back in 2009, there was some development around Google Indic Transliteration which supported about 13 major languages from around the Indian sub-continent- http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5621. However, it ultimately failed QA and was not available. In 2011, Google Transliteration API was deprecated, and while it is still available, the Google ToS do not extend well towards a Free Software project. So, I revisited the jQuery.IME project[1] again. It has matured a lot since then with over 135 input methods spread over 67 languages[2]. This count is growing as the library is key to Wikimedia's UniversalLanguageSelector[3] and VisualEditor stack[4]. The best part of this patch is that it doesn't add anything persistent on the viewport real estate. The code hooks into input fields of text / textarea types and become available to the user on-demand. The library remains dual-licensed under GPLv2+ and MIT. Reference: [1] https://github.com/wikimedia/jquery.ime [2] http://www.mediawiki.org/wiki/Milkshake/status [3] http://www.mediawiki.org/wiki/Extension:UniversalLanguageSelector [4] http://www.mediawiki.org/wiki/VisualEditor
*** Bug 9515 has been marked as a duplicate of this bug. ***
Created attachment 31152 [details] [review] Adds support of multilingual input methods to Koha This patch add a minified jQuery.IME library along with its rule files. Next, it add the necessary sysprefs and code to handle user-driven activation or otherwise of jquery.ime suppor inside Koha's OPAC and staff client.
This patch integrates Wikimedia's jQuery.IME multi-lingual input method jQuery based library with Koha, for the OPAC as well as the staff client. Browser based on-the-fly transliterated input on the OPAC was a long-standing requirement particularly in the Indian Koha user community. Indian government recognises 22 official languages for the purpose of official correspondance. The earlier attempts to add GoogleIndicTransliteration was a step in this direction. But that attempt did not clear finally QA. Google Transliteration API was deprecated in June 2011 and it's general Terms of Service (ToS) are not FOSS friendly. In contrast, jQuery.IME is dual-licensed under GPLv2+ and MIT. It is a key component of Wikimedia's global language engineering stack. At present, this represents robust, well tested support for over 135 input methods spanning over 67 languages. Test Plan: 1) Install the patch. 2) Update the database (so that the new system preferences are inserted) 3) Go to Administration -> System Preferences -> I18N/L10N in the staff client. 4) Under "Multi-Lingual keyboard", "Enable" the UsejQueryIMEonOPAC system preference. Open the OPAC, and click inside the search box after page load. You should see small keyboard graphics attached to the bottom right edge of the search box. 5) Click on the keyboard graphics to expand the drop-down. Select your language. The drop-down will close. 6) Click again on the keyboard icon, it should now offer the list of input methods available for the language selected in step 5. 7) You should be able to type in your chosen input method. To switch back to English while on that page press CTRL+M. This toggles between input methods while on the page. 8) Repeating step #4 about to select "Do not enable", will disable the jQuery.IME support. 9) Repeat steps #3 to #8 with the other syspref UsejQueryIMEonStaff and check accordingly in the Staff Client interface. The results should be same as with the OPAC. NOTE 1: For a language to work, the operating system should have UTF-8 support and in the case of Indic and other asian languages Complex layout rendering support and the necessary fonts. NOTE 2: A set of screenshot showing steps 5, 6 and 7 using the OPAC. (a) Default view with IM active - http://imgur.com/a/2gB7H#3 (b) Selecting the language - http://imgur.com/a/2gB7H#1 (c) Search box showing input method - http://imgur.com/a/2gB7H#0 (d) Typing using Hindi Inscript im - http://imgur.com/a/2gB7H#2
This appears to be working correctly for me, but I have a couple of comments: 1. There should not be comments added to the markup referencing the bug number. We have git for tracking changes. 2. Instead of adding two new include files, why not add that template logic to doc-head-close.inc? I think the creation of new include files is only useful if they will be included by multiple templates. The functionality of the plugin itself seems to be good, but I think it should be tested again by someone whose native language is represented in the list of options.
One more thing: the plugin details should be added to the staff client's about page (about.tt).
(In reply to Owen Leonard from comment #5) > One more thing: the plugin details should be added to the staff client's > about page (about.tt). Will add and post patch immediately.
(In reply to Owen Leonard from comment #4) > This appears to be working correctly for me, but I have a couple of comments: > > 1. There should not be comments added to the markup referencing the bug > number. We have git for tracking changes. Will remove and update. > 2. Instead of adding two new include files, why not add that template logic > to doc-head-close.inc? I think the creation of new include files is only > useful if they will be included by multiple templates. For, the staff client, I think I'll have to introduce [% USE Koha %] into doc-head-open.inc > The functionality of the plugin itself seems to be good, but I think it > should be tested again by someone whose native language is represented in > the list of options. that makes sense. @Amit: Can you please test it out?
Created attachment 31173 [details] [review] Adds [% USE Koha %] TT plugin on intranet templates missing them The original patch http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31152 is being split up into a 5-part patchset as per discussion on #koha This 1/5 adds the [% USE Koha %] TT plugin at the first line in each of 508 out of 523.tt files placed under /intranet-tmpl/prog/en/modules. The remaining 15 files were already using it. Test Plan: Patch 1 of 5 do not have a separate stand-alone test plan. The patch is to be applied as the first patch for bug 12815. The overall test plan is defined later in patches 4 and 5 in the set.
Created attachment 31174 [details] [review] Adds jQuery.IME library under intranet-tmpl/lib for staff client The original patch http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31152 is being split up into a 5-part patchset as per discussion on #koha This 2/5 adds the jQuery.IME library for inclusion by the staff client code along with licensing information of the library in about.tt Test Plan: Patch 2 of 5 do not have a separate stand-alone test plan. The patch is to be applied as the second patch for bug 12815. The overall test plan is defined later in patches 4 and 5 in the set.
Created attachment 31175 [details] [review] Adds jQuery.IME library under opac-tmpl/lib for OPAC The original patch http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31152 is being split up into a 5-part patchset as per discussion on #koha This 3/5 adds the jQuery.IME library for inclusion by the OPAC code. Test Plan: Patch 3 of 5 do not have a separate stand-alone test plan. The patch is to be applied as the third patch for bug 12815. The overall test plan is defined later in patches 4 and 5 in the set.
Created attachment 31176 [details] [review] Adds the jQuery.IME support hooks for the Koha OPAC The original patch http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31152 is being split up into a 5-part patchset as per discussion on #koha This 4/5 adds the jQuery.IME support hooks for the Koha OPAC. It introduces a new syspref 'UsejQueryIMEonOPAC' and inserts into sysprefs.sql and updatedatabase.pl. The i18n_l10n.pref YAML file is also organised into topical sub-sections i.e. 'Date and Time', 'Language Support' and 'Multi-lingual keyboard'. UsejQueryIMEonOPAC syspref is added under 'Multi-lingual keyboard' sub-section Test Plan: 1) Apply patches 2) Update the database (so that the new system preference is inserted) 3) Go to Administration -> System Preferences -> I18N/L10N in the staff client. 4) Under "Multi-Lingual keyboard", "Enable" the UsejQueryIMEonOPAC system preference. Open the OPAC, and click inside the search box after page load. You should see small keyboard graphics attached to the bottom right edge of the search box. 5) Click on the keyboard graphics to expand the drop-down. Select your language. The drop-down will close. 6) Click again on the keyboard icon, it should now offer the list of input methods available for the language selected in step 5. 7) You should be able to type in your chosen input method. To switch back to English while on that page press CTRL+M. This toggles between input methods while on the page. 8) Repeating step #4 about to select "Do not enable", should disable jQuery.IME support. NOTE 1: For a language to work, the operating system should have UTF-8 support and in the case of Indic and other asian languages Complex layout rendering support and the necessary fonts. NOTE 2: A set of screenshot showing steps 5, 6 and 7 using the OPAC. (a) Default view with IM active - http://imgur.com/a/2gB7H#3 (b) Selecting the language - http://imgur.com/a/2gB7H#1 (c) Search box showing input method - http://imgur.com/a/2gB7H#0 (d) Typing using Hindi Inscript im - http://imgur.com/a/2gB7H#2
Created attachment 31177 [details] [review] Adds the jQuery.IME support hooks for the Koha staff client The original patch http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31152 is being split up into a 5-part patchset as per discussion on #koha This 5/5 adds the jQuery.IME support hooks for the Koha staff client. It introduces a new syspref 'UsejQueryIMEonStaff' and inserts into sysprefs.sql and updatedatabase.pl. The entry for UsejQueryIMEonStaff is also added in the i18n_l10n.pref YAML file under 'Multi-lingual keyboard' sub-section. Test Plan: 1) Apply patch 2) Update the database (so that the new system preference is inserted) 3) Go to Administration -> System Preferences -> I18N/L10N in the staff client. 4) Under "Multi-Lingual keyboard", "Enable" the UsejQueryIMEonStaff system preference. Open the refresh the staff client window and click inside any input text or textarea field after the page loads. You should see small keyboard graphics attached to the bottom right edge of the text field. 5) Click on the keyboard graphics to expand the drop-down. Select your language. The drop-down will close. 6) Click again on the keyboard icon, it should now offer the list of input methods available for the language selected in step 5. 7) You should be able to type in your chosen input method. To switch back to English while on that page press CTRL+M. This toggles between input methods while on the page. 8) Repeating step #4 about to select "Do not enable", should disable jQuery.IME support for the staff client. NOTE 1: For a language to work, the operating system should have UTF-8 support and in the case of Indic and other asian languages Complex layout rendering support and the necessary fonts.
Created attachment 31179 [details] [review] Adds [% USE Koha %] TT plugin on intranet templates missing them The original patch http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31152 is being split up into a 5-part patchset as per discussion on #koha This 1/5 adds the [% USE Koha %] TT plugin at the first line in each of 508 out of 523 .tt files placed under /intranet-tmpl/prog/en/modules. The remaining 15 files were already using it. Test Plan: Patch 1 of 5 do not have a separate stand-alone test plan. The patch is to be applied as the first patch for bug 12815. The overall test plan is defined later in patches 4 and 5 in the set.
The really long [% USE Koha %] patch is obsoleted. It should be [%- USE Koha -%] to avoid spacing issues. But don't put that change in this bug. :) However, this patch does apply, if you have merged the latest changes including Bug 10212. I'm changing this to 'Needs Sign Off'. The Test Plan is in comment #11.
Created attachment 31218 [details] [review] Adds the jQuery.IME library for staff client
Created attachment 31219 [details] [review] Adds the jQuery.IME library for OPAC
Created attachment 31220 [details] [review] Adds jQuery.IME support to the OPAC
Created attachment 31221 [details] [review] Adds jQuery.IME support to the staff client
Comment on attachment 31220 [details] [review] Adds jQuery.IME support to the OPAC Third patch in the set and should be applied in that sequence. Adds UsejQueryIMEonOPAC syspref and updates sysprefs.sql and updatedatabase.pl accordingly. Also updates i18n_l10n.pref, by introducing sub-sections - Date and Time, Language Support and Multi-lingual keyboard; with UsejQueryIMEonOPAC syspref being added under Multi-lingual keyboard sub-section. Test plan 1) Apply patches 2) Update the database (so that the new system preference is inserted) 3) Go to Administration -> System Preferences -> I18N/L10N in the staff client. 4) Under "Multi-Lingual keyboard", "Enable" the UsejQueryIMEonOPAC system preference. Open the OPAC, and click inside the search box after page load. You should see small keyboard graphics attached to the bottom right edge of the search box. 5) Click on the keyboard graphics to expand the drop-down. Select your language. The drop-down will close. 6) Click again on the keyboard icon, it should now offer the list of input methods available for the language selected in step 5. 7) You should be able to type in your chosen input method. To switch back to English while on that page press CTRL+M. This toggles between input methods while on the page. 8) Repeating step #4 about to select "Do not enable", should disable jQuery.IME support. NOTE 1: For a language to work, the operating system should have UTF-8 support and in the case of Indic and other asian languages Complex layout rendering support and the necessary fonts. NOTE 2: A set of screenshot showing steps 5, 6 and 7 using the OPAC. (a) Default view with IM active - http://imgur.com/a/2gB7H#3 (b) Selecting the language - http://imgur.com/a/2gB7H#1 (c) Search box showing input method - http://imgur.com/a/2gB7H#0 (d) Typing using Hindi Inscript im - http://imgur.com/a/2gB7H#2
Quick scan of the patch. You really need to put this into the bootstrap theme, not only the prog theme for master. And with the deprecation of the prog theme, you may wish to split the OPAC patch for it apart from the bootstrap portion. Keep it up. It will get in there with enough cleanup. :)
Created attachment 31223 [details] [review] Bug 12815 [ENH] Patch 4 of 4 - Add jQuery.IME support to staff client Fourth patch in the set and should be applied in that sequence. Adds UsejQueryIMEonStaff syspref and updates sysprefs.sql and updatedatabase.pl accordingly. Also updates i18n_l10n.pref, by adding UsejQueryIMEonStaff under Multi-lingual keyboard sub-section. Test Plan: 1) Apply patch 2) Update the database (so that the new system preference is inserted) 3) Go to Administration -> System Preferences -> I18N/L10N in the staff client. 4) Under "Multi-Lingual keyboard", "Enable" the UsejQueryIMEonStaff system preference. Open the refresh the staff client window and click inside any input text or textarea field after the page loads. You should see small keyboard graphics attached to the bottom right edge of the text field. 5) Click on the keyboard graphics to expand the drop-down. Select your language. The drop-down will close. 6) Click again on the keyboard icon, it should now offer the list of input methods available for the language selected in step 5. 7) You should be able to type in your chosen input method. To switch back to English while on that page press CTRL+M. This toggles between input methods while on the page. 8) Repeating step #4 about to select "Do not enable", should disable jQuery.IME support for the staff client. NOTE 1: For a language to work, the operating system should have UTF-8 support and in the case of Indic and other asian languages Complex layout rendering support and the necessary fonts. Followed test plan. Behaves as expected. Small typo in preferences (i18n_l10n.pref): It should read "Do not enable" (missing t). Can this be fixed with a follow up? Signed-off-by: Marc Véron <veron@veron.ch>
Sorry, I missed M.Tompsetts comment. Remarks: 1) Small typo in preferences (i18n_l10n.pref): It should read "Do not enable" (missing t). 2) koha-tmpl/intranet-tmpl/prog/en/modules/about.tt Text about jQuery Colvis plugin is removed by first patch. Is this by intention?
(In reply to M. Tompsett from comment #20) > Quick scan of the patch. You really need to put this into the bootstrap > theme, not only the prog theme for master. Did not understand ^^^ :-( The staff client code touches this file -> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31221&action=diff#a/koha-tmpl/intranet-tmpl/prog/en/includes/doc-head-close.inc_sec1 The OPAC code touches : 1) http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31220&action=diff#a/koha-tmpl/opac-tmpl/bootstrap/en/includes/doc-head-close.inc_sec1 which is within bootstrap. and 2) http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31221&action=diff#a/koha-tmpl/intranet-tmpl/prog/en/includes/doc-head-close.inc_sec2 also within bootstrap. If I was supposed to do differently, please guide me. > Keep it up. It will get in there with enough cleanup. :) thanks :-)
(In reply to Indranil Das Gupta from comment #23) > (In reply to M. Tompsett from comment #20) > > Quick scan of the patch. You really need to put this into the bootstrap > > theme, not only the prog theme for master. > > Did not understand ^^^ :-( > 2) > http://bugs.koha-community.org/bugzilla3/attachment. > cgi?id=31221&action=diff#a/koha-tmpl/intranet-tmpl/prog/en/includes/doc-head- > close.inc_sec2 also within bootstrap. aargh! I see! mea culpa! will fix ASAP!
(In reply to Indranil Das Gupta from comment #24) > (In reply to Indranil Das Gupta from comment #23) > > (In reply to M. Tompsett from comment #20) > > > Quick scan of the patch. You really need to put this into the bootstrap > > > theme, not only the prog theme for master. > > > > Did not understand ^^^ :-( > > > 2) > > http://bugs.koha-community.org/bugzilla3/attachment. > > cgi?id=31221&action=diff#a/koha-tmpl/intranet-tmpl/prog/en/includes/doc-head- > > close.inc_sec2 also within bootstrap. > > aargh! I see! mea culpa! will fix ASAP! Umm.... this one is NOT_A_PROB. I'm seeing too many wrong things today. the second file is http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31220&action=diff#a/koha-tmpl/opac-tmpl/bootstrap/en/includes/opac-bottom.inc_sec1 also within bootstrap
Created attachment 31231 [details] [review] Correct typo in UsejQueryIMEonOPAC syspref in i18n_l10n.pref Corrects the typo "Do no enable" to "Do not enable" for UsejQueryIMEonOPAC syspref definition in i18n_l10n.pref.
Created attachment 31232 [details] [review] Correct typo in UsejQueryIMEonStaff syspref in i18n_l10n.pref Corrects the typo "Do no enable" to "Do not enable" for UsejQueryIMEonStaff syspref definition in i18n_l10n.pref.
Created attachment 31233 [details] [review] Fixes accidental deletion of jQuery Colvis license in about.tt Fixes accidental deletion of jQuery Colvis license information in about.tt
(In reply to Marc Véron from comment #22) > Small typo in preferences (i18n_l10n.pref): It should read "Do not enable" > (missing t). Fixed. See attachment 31231 [details] [review] and attachment 31232 [details] [review] > koha-tmpl/intranet-tmpl/prog/en/modules/about.tt > Text about jQuery Colvis plugin is removed by first patch. Is this by > intention? Fixed. Not intentional, bad case of over-pasting. See attachment 31233 [details] [review]. Please re-test and sign-off when convenient.
Created attachment 31238 [details] [review] Fixes accidental deletion of jQuery Colvis license in about.tt
Sorry, the following patch does not apply: Applying: Bug 12815 [ENH] Fixing accidental deletion of jQuery Colvis license info fatal: sha1 information is lacking or useless (koha-tmpl/intranet-tmpl/prog/en/modules/about.tt). Repository lacks necessary blobs to fall back on 3-way merge. Cannot fall back to three-way merge.
(In reply to Marc Véron from comment #31) > Sorry, the following patch does not apply: > > Applying: Bug 12815 [ENH] Fixing accidental deletion of jQuery Colvis > license info > fatal: sha1 information is lacking or useless > (koha-tmpl/intranet-tmpl/prog/en/modules/about.tt). > Repository lacks necessary blobs to fall back on 3-way merge. > Cannot fall back to three-way merge. Did you apply this without applying the first patch which replaced it accidentally? I had no problems applying this entire set of patches.
I did: git bz apply 12815 ...and got the following: Bug 12815 - Adding browser based multi-lingual jquery.ime input method support to Koha 31218 - Adds the jQuery.IME library for staff client 31219 - Adds the jQuery.IME library for OPAC 31220 - Adds jQuery.IME support to the OPAC 31221 - Adds jQuery.IME support to the staff client 31231 - Correct typo in UsejQueryIMEonOPAC syspref in i18n_l10n.pref 31232 - Correct typo in UsejQueryIMEonStaff syspref in i18n_l10n.pref 31233 - Fixes accidental deletion of jQuery Colvis license in about.tt Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 12815 [ENH] Patch 1 of 4 - Adds jquery.ime libs for staff client Applying: Bug 12815 [ENH] Patch 2 of 4 - Add jquery.ime lib for staff client Applying: Bug 12815 [ENH] Patch 3 of 4 - Add jQuery.IME support to OPAC Applying: Bug 12815 [ENH] Patch 4 of 4 - Add jQuery.IME support to staff client Applying: Bug 12815 [ENH] Removes UsejQueryIMEonOPAC syspref typo in i18n_l10n.pref Applying: Bug 12815 [ENH] Removes UsejQueryIMEonStaff syspref typo in i18n_l10n.pref Applying: Bug 12815 [ENH] Fixing accidental deletion of jQuery Colvis license info fatal: sha1 information is lacking or useless (koha-tmpl/intranet-tmpl/prog/en/modules/about.tt). Repository lacks necessary blobs to fall back on 3-way merge. Cannot fall back to three-way merge. Patch failed at 0001 Bug 12815 [ENH] Fixing accidental deletion of jQuery Colvis license info When you have resolved this problem run "git bz apply --continue". If you would prefer to skip this patch, instead run "git bz apply --skip". To restore the original branch and stop patching run "git bz apply --abort". Patch left in /tmp/Fixes-accidental-deletion-of-jQuery-Colvis-license-dAwEUs.patch
(In reply to Marc Véron from comment #33) > I did: > git bz apply 12815 > > ...and got the following: [SNIPPED UGLY MESS] Is your master up to date? I just reattempted on my ancient UNIMARC VM. $ git pull $ git reset --hard origin/master $ git bz apply 12815 worked for me.
Yes, I did... git pull Already up-to-date. git reset --hard origin/master HEAD is now at 4e1aaa4 Revert "Bug 12176 - Remove HTML from additem.pl" git bz apply 1285 Fails as above...
Is your apt sources squeeze or squeeze-dev? Mine is the latter. Because other than that I can't think of any other reason you would get that weird sha1 problem. When's the last time you did a git fsck?
I can apply these patches without any trouble using git bz.
Dear Indra, I appreciate your effort. Can you extend the keyboard to cataloguing framework? The keyboard work without Internet?
(In reply to vimal kumar from comment #38) > Can you extend the keyboard to cataloguing framework? I do not understand your question here. Attachment 31220 [details] and Attachment 31221 [details] adds the support to the OPAC and the Staff Client respectively. Can you explain? > The keyboard work without Internet? Yes, the necessary js libs are included. Being online is not a requirement.
(In reply to vimal kumar from comment #38) > Can you extend the keyboard to cataloguing framework? Did you mean the value builder for 008 - marc21_field_008.pl? As mtompset pointed out on IRC, that one does not include the necessary .inc file.
I meant that virtual keyboard in data entry sheet is helpful while entering book details in regional language.
Then I should apologize, Indranil. I think I suggested a wild goose chase. :)
(In reply to vimal kumar from comment #41) > I meant that virtual keyboard in data entry sheet is helpful while entering > book details in regional language. I don't know what "data entry sheet" refers to, but the MARC editor (addbiblio.pl) includes the virtual keyboard. However, I do find that with this feature enabled the MARC editor page loads very slowly. Firefox almost crashed, Chrome popped up a "page unresponsive" dialog. It looks like there may be an upper limit to the number of form fields the plugin can handle.
(In reply to M. Tompsett from comment #42) > Then I should apologize, Indranil. I think I suggested a wild goose chase. :) Naah! that was a fair suggestion, 008 does indeed lack the support. So, it is still pertinent IMHO :)
(In reply to Owen Leonard from comment #43) > (In reply to vimal kumar from comment #41) [snipped] > However, I do find that with this feature enabled the MARC editor page loads > very slowly. Firefox almost crashed, Chrome popped up a "page unresponsive" > dialog. It looks like there may be an upper limit to the number of form > fields the plugin can handle. Owen, aah! I suspected as much. I faced similar behaviour when running it off my VPS a few times. I found this to be intermittent. But it did not seem to happen on a local install. In the meanwhile, I'll go pick the brains of the Wikimedia Language Engg team as to the cause and its redress.
I'm switching this back to "assigned" based on the last comment.