A library requested the ability to add emojis in the tag field on the opac. Entering an emoji now seems to scrub it clear and leave no value in the db but still enter a new tag. Subsequent attempts warn of repeated tags.
mysql> show variables like '%char%'; +--------------------------+----------------------------+ | Variable_name | Value | +--------------------------+----------------------------+ | character_set_client | utf8 | | character_set_connection | utf8 | | character_set_database | latin1 | | character_set_filesystem | binary | | character_set_results | utf8 | | character_set_server | latin1 | | character_set_system | utf8 | | character_sets_dir | /usr/share/mysql/charsets/ | +--------------------------+----------------------------+ 8 rows in set (0.00 sec) mysql> set character_set_database=utf8mb4; Query OK, 0 rows affected (0.00 sec) mysql> set character_set_connection=utf8mb4; Query OK, 0 rows affected (0.00 sec) mysql> set character_set_client=utf8mb4; Query OK, 0 rows affected (0.00 sec) mysql> set character_set_results=utf8mb4; Query OK, 0 rows affected (0.00 sec) mysql> show variables like '%char%'; +--------------------------+----------------------------+ | Variable_name | Value | +--------------------------+----------------------------+ | character_set_client | utf8mb4 | | character_set_connection | utf8mb4 | | character_set_database | utf8mb4 | | character_set_filesystem | binary | | character_set_results | utf8mb4 | | character_set_server | latin1 | | character_set_system | utf8 | | character_sets_dir | /usr/share/mysql/charsets/ | +--------------------------+----------------------------+ 8 rows in set (0.00 sec) mysql> drop database test_unicode; Query OK, 1 row affected (0.01 sec) mysql> CREATE DATABASE test_unicode DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) mysql> use test_unicode; Database changed mysql> create table simple_table (text varchar(100)) collate utf8_unicode_ci; Query OK, 0 rows affected (0.01 sec) mysql> INSERT INTO simple_table VALUES ('[U+1F4A9]'); Query OK, 1 row affected, 1 warning (0.00 sec) mysql> show warnings; +---------+------+-----------------------------------------------------------------------+ | Level | Code | Message | +---------+------+-----------------------------------------------------------------------+ | Warning | 1366 | Incorrect string value: '\xF0\x9F\x92\xA9' for column 'text' at row 1 | +---------+------+-----------------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> delete from simple_table; Query OK, 1 row affected (0.00 sec) mysql> drop table simple_table; Query OK, 0 rows affected (0.00 sec) mysql> create table simple_table (text varchar(100)) collate utf8mb4_unicode_ci; Query OK, 0 rows affected (0.02 sec) mysql> INSERT INTO simple_table VALUES ('[U+1F4A9]'); Query OK, 1 row affected (0.01 sec) mysql> select * from simple_table; +------+ | text | +------+ | [U+1F4A9] | +------+ 1 row in set (0.00 sec) mysql>
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8mb4.html Hope these point in a useful direction.
P.S. DataTables may not be as unicode friend as we would like for this.
See bug 11944 comment 247
*** This bug has been marked as a duplicate of bug 15885 ***
*** Bug 15885 has been marked as a duplicate of this bug. ***
Work to be done: * Fix storage of Emoji characters in the database ** Emoji characters are supported in Unicode: http://www.unicode.org/emoji/charts/full-emoji-list.html * Research and implement tools to display Emojis properly in the Staff and OPAC Seeking co-sponsors: http://devs.bywatersolutions.com/projects/enable-emoji-support-tags/
I made this bug dependent on 18336, because (as Mark correctly pointed out) the big problem for this is the DB handling this 4-byte characters (emojis, some japanese chars, etc). Changing the DB like this is important for other areas, not just tags. Luckily, I've submitted the patches to implement the change (on bug 18336). It will need to be broadly tested, so start now :-D I will leave this bug open, for the specific task of integrating an emoji picker for the tags adding form in the OPAC :-D
I've had trouble posting the patches here. Please use this branch to get the patches: https://github.com/tomascohen/koha/tree/bug_15794_emoji_tags
I would recommend posting just the non-emoji picker parts, and then having people do the following on a kohadevbox: cd ~ git clone https://github.com/OneSignal/emoji-picker.git cd emoji-picker/lib mkdir ~/kohaclone/koha-tmpl/opac-tmpl/lib/emoji-picker cp -r -v . ~/kohaclone/koha-tmpl/opac-tmpl/lib/emoji-picker cd ~/kohaclone git add `find ~/kohaclone/koha-tmpl/opac-tmpl/lib/emoji-picker` git commit
Created attachment 71053 [details] [review] Bug 15794: Add Emoji Picker TEST PLAN --------- 1) apply this patch 2) Run the commands in comment #10 3) Attempt to add emoji tags in OPAC -- some system preferences may need setting 4) run koha qa test tools NOTE: The failure resulting from the click will be a different bug. Also, do not include the emoji picker library, as the PNG files are too large to attach. Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Remember to apply the dependency first, and run an update, and perhaps even a reset_all just to make sure your data in prepared for this.
Additionally follow up bug that needs creating, updating the about area with the list of One Signal emoji-picker under the MIT and Apache licenses.
(In reply to M. Tompsett from comment #13) > Additionally follow up bug that needs creating, updating the about area with > the list of One Signal emoji-picker under the MIT and Apache licenses. Mark, I will add your signature to the version in my github. BTW, I've just updated the patch so the emoji picker works on the results page too!
https://github.com/bywatersolutions/bywater-koha-devel/commit/c30f482d798b5402c357024f318c030c1e26bec0
Created attachment 71860 [details] [review] Bug 15794: Add [% KOHA_VERSION to .css and .js
Created attachment 71861 [details] [review] Bug 15794: Add emojiarea to the license area of the about page
Pushed to master for 18.05, thanks to everybody involved!
The emoji picker is causing script errors when using Koha in Endnote X9. I think Endnote X9 uses an (old) embedded version of Internet Explorer, and I think that's why it's happening. Investigating ways of fixing it, but I think I might just send a follow-up about using a system preference to disable the emoji picker.
(In reply to David Cook from comment #19) > I think Endnote X9 uses an (old) embedded version of Internet Explorer A scary Halloween short story as told by David.
(In reply to David Cook from comment #19) > The emoji picker is causing script errors when using Koha in Endnote X9. > > I think Endnote X9 uses an (old) embedded version of Internet Explorer, and > I think that's why it's happening. > > Investigating ways of fixing it, but I think I might just send a follow-up > about using a system preference to disable the emoji picker. +1 for a syspref. Or better find a more compatible emoji picker. Anyways, please default to enabled to keep the current behavior.
(In reply to Owen Leonard from comment #20) > (In reply to David Cook from comment #19) > > > I think Endnote X9 uses an (old) embedded version of Internet Explorer > > A scary Halloween short story as told by David. [U+1F923] I love you, Owen.
(In reply to Tomás Cohen Arazi from comment #21) > > +1 for a syspref. Or better find a more compatible emoji picker. > > Anyways, please default to enabled to keep the current behavior. Sounds good to me. For now, we've just hacked it away, but I'll keep this in the back of my mind.