Bug 15794

Summary: Add emoji picker to tag entry in OPAC
Product: Koha Reporter: Nick Clemens <nick>
Component: OPACAssignee: Tomás Cohen Arazi <tomascohen>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: dcook, jonathan.druart, katrin.fischer, kyle, mtj, mtompset, rhilliker, tomascohen, veron
Version: Main   
Hardware: All   
OS: All   
URL: http://devs.bywatersolutions.com/projects/enable-emoji-support-tags/
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11944
Change sponsored?: Seeking cosponsors Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 18336    
Bug Blocks: 25350    
Attachments: Bug 15794: Add Emoji Picker
Bug 15794: Add [% KOHA_VERSION to .css and .js
Bug 15794: Add emojiarea to the license area of the about page

Description Nick Clemens 2016-02-10 22:05:18 UTC
A library requested the ability to add emojis in the tag field on the opac.

Entering an emoji now seems to scrub it clear and leave no value in the db but still enter a new tag.  Subsequent attempts warn of repeated tags.
Comment 1 Mark Tompsett 2016-02-18 02:27:01 UTC
mysql> show variables like '%char%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | latin1                     |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | latin1                     |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

mysql> set character_set_database=utf8mb4;
Query OK, 0 rows affected (0.00 sec)

mysql> set character_set_connection=utf8mb4;
Query OK, 0 rows affected (0.00 sec)

mysql> set character_set_client=utf8mb4;
Query OK, 0 rows affected (0.00 sec)

mysql> set character_set_results=utf8mb4;
Query OK, 0 rows affected (0.00 sec)

mysql> show variables like '%char%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8mb4                    |
| character_set_connection | utf8mb4                    |
| character_set_database   | utf8mb4                    |
| character_set_filesystem | binary                     |
| character_set_results    | utf8mb4                    |
| character_set_server     | latin1                     |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

mysql> drop database test_unicode;
Query OK, 1 row affected (0.01 sec)

mysql> CREATE DATABASE test_unicode DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)

mysql> use test_unicode;
Database changed
mysql> create table simple_table (text varchar(100)) collate utf8_unicode_ci;
Query OK, 0 rows affected (0.01 sec)

mysql> INSERT INTO simple_table VALUES ('💩');
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> show warnings;
+---------+------+-----------------------------------------------------------------------+
| Level   | Code | Message                                                               |
+---------+------+-----------------------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\xF0\x9F\x92\xA9' for column 'text' at row 1 |
+---------+------+-----------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> delete from simple_table;
Query OK, 1 row affected (0.00 sec)

mysql> drop table simple_table;
Query OK, 0 rows affected (0.00 sec)

mysql> create table simple_table (text varchar(100)) collate utf8mb4_unicode_ci;
Query OK, 0 rows affected (0.02 sec)

mysql> INSERT INTO simple_table VALUES ('💩');
Query OK, 1 row affected (0.01 sec)

mysql> select * from simple_table;
+------+
| text |
+------+
| 💩     |
+------+
1 row in set (0.00 sec)

mysql>
Comment 2 Mark Tompsett 2016-02-18 02:29:23 UTC
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8mb4.html

Hope these point in a useful direction.
Comment 3 Mark Tompsett 2016-02-18 06:32:56 UTC
P.S. DataTables may not be as unicode friend as we would like for this.
Comment 4 Jonathan Druart 2016-02-18 11:57:44 UTC
See bug 11944 comment 247
Comment 5 Nicole C. Engard 2016-02-23 16:57:55 UTC

*** This bug has been marked as a duplicate of bug 15885 ***
Comment 6 Katrin Fischer 2016-02-29 21:30:06 UTC
*** Bug 15885 has been marked as a duplicate of this bug. ***
Comment 7 Nicole C. Engard 2016-03-02 16:15:46 UTC
Work to be done:

* Fix storage of Emoji characters in the database

 ** Emoji characters are supported in Unicode: http://www.unicode.org/emoji/charts/full-emoji-list.html

* Research and implement tools to display Emojis properly in the Staff and OPAC

Seeking co-sponsors: http://devs.bywatersolutions.com/projects/enable-emoji-support-tags/
Comment 8 Tomás Cohen Arazi 2018-01-23 23:35:04 UTC
I made this bug dependent on 18336, because (as Mark correctly pointed out) the big problem for this is the DB handling this 4-byte characters (emojis, some japanese chars, etc). Changing the DB like this is important for other areas, not just tags.

Luckily, I've submitted the patches to implement the change (on bug 18336). It will need to be broadly tested, so start now :-D

I will leave this bug open, for the specific task of integrating an emoji picker for the tags adding form in the OPAC :-D
Comment 9 Tomás Cohen Arazi 2018-01-29 13:07:41 UTC
I've had trouble posting the patches here. Please use this branch to get the patches:

https://github.com/tomascohen/koha/tree/bug_15794_emoji_tags
Comment 10 Mark Tompsett 2018-01-29 20:16:33 UTC
I would recommend posting just the non-emoji picker parts, and then having people do the following on a kohadevbox:
    cd ~
    git clone https://github.com/OneSignal/emoji-picker.git
    cd emoji-picker/lib
    mkdir ~/kohaclone/koha-tmpl/opac-tmpl/lib/emoji-picker
    cp -r -v . ~/kohaclone/koha-tmpl/opac-tmpl/lib/emoji-picker
    cd ~/kohaclone
    git add `find ~/kohaclone/koha-tmpl/opac-tmpl/lib/emoji-picker`
    git commit
Comment 11 Mark Tompsett 2018-01-30 01:11:17 UTC
Created attachment 71053 [details] [review]
Bug 15794: Add Emoji Picker

TEST PLAN
---------
1) apply this patch
2) Run the commands in comment #10
3) Attempt to add emoji tags in OPAC
   -- some system preferences may need setting
4) run koha qa test tools

NOTE: The failure resulting from the click will be
      a different bug.
      Also, do not include the emoji picker library,
      as the PNG files are too large to attach.

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Comment 12 Mark Tompsett 2018-01-30 01:12:17 UTC
Remember to apply the dependency first, and run an update, and perhaps even a reset_all just to make sure your data in prepared for this.
Comment 13 Mark Tompsett 2018-01-30 01:27:31 UTC
Additionally follow up bug that needs creating, updating the about area with the list of One Signal emoji-picker under the MIT and Apache licenses.
Comment 14 Tomás Cohen Arazi 2018-01-30 14:57:48 UTC
(In reply to M. Tompsett from comment #13)
> Additionally follow up bug that needs creating, updating the about area with
> the list of One Signal emoji-picker under the MIT and Apache licenses.

Mark, I will add your signature to the version in my github. BTW, I've just updated the patch so the emoji picker works on the results page too!
Comment 16 Jonathan Druart 2018-02-16 20:48:44 UTC
Created attachment 71860 [details] [review]
Bug 15794: Add [% KOHA_VERSION to .css and .js
Comment 17 Jonathan Druart 2018-02-16 20:48:50 UTC
Created attachment 71861 [details] [review]
Bug 15794: Add emojiarea to the license area of the about page
Comment 18 Jonathan Druart 2018-02-16 21:17:52 UTC
Pushed to master for 18.05, thanks to everybody involved!
Comment 19 David Cook 2019-03-18 04:04:01 UTC
The emoji picker is causing script errors when using Koha in Endnote X9. 

I think Endnote X9 uses an (old) embedded version of Internet Explorer, and I think that's why it's happening. 

Investigating ways of fixing it, but I think I might just send a follow-up about using a system preference to disable the emoji picker.
Comment 20 Owen Leonard 2019-03-18 16:06:09 UTC
(In reply to David Cook from comment #19)

> I think Endnote X9 uses an (old) embedded version of Internet Explorer

A scary Halloween short story as told by David.
Comment 21 Tomás Cohen Arazi 2019-03-18 16:29:23 UTC
(In reply to David Cook from comment #19)
> The emoji picker is causing script errors when using Koha in Endnote X9. 
> 
> I think Endnote X9 uses an (old) embedded version of Internet Explorer, and
> I think that's why it's happening. 
> 
> Investigating ways of fixing it, but I think I might just send a follow-up
> about using a system preference to disable the emoji picker.

+1 for a syspref. Or better find a more compatible emoji picker.

Anyways, please default to enabled to keep the current behavior.
Comment 22 David Cook 2019-03-25 03:59:53 UTC
(In reply to Owen Leonard from comment #20)
> (In reply to David Cook from comment #19)
> 
> > I think Endnote X9 uses an (old) embedded version of Internet Explorer
> 
> A scary Halloween short story as told by David.

🤣

I love you, Owen.
Comment 23 David Cook 2019-03-25 04:01:10 UTC
(In reply to Tomás Cohen Arazi from comment #21)
>
> +1 for a syspref. Or better find a more compatible emoji picker.
> 
> Anyways, please default to enabled to keep the current behavior.

Sounds good to me. For now, we've just hacked it away, but I'll keep this in the back of my mind.