With COLLATE=utf8_unicode_ci it's not possible to map fields with same letter but different case (eg 995a and 995A) with the form of elasticsearch/mappings.pl. Changing the collation in the database fix the issue.
I confirm, it's still _ci: -- -- Table structure for table 'search_marc_map' -- DROP TABLE IF EXISTS search_marc_map; CREATE TABLE `search_marc_map` ( id int(11) NOT NULL AUTO_INCREMENT, index_name ENUM('biblios','authorities') NOT NULL COMMENT 'what storage index this map is for', marc_type ENUM('marc21', 'unimarc', 'normarc') NOT NULL COMMENT 'what MARC type this map is for', marc_field VARCHAR(255) NOT NULL COMMENT 'the MARC specifier for this field', PRIMARY KEY(`id`), UNIQUE key `index_name` (`index_name`, `marc_field` (191), `marc_type`), INDEX (`index_name`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; As we are also using uppercase letters to extent the item subfields, I'd be interested in fixing this. Note: we already have the different collation on marc_subfield_structure.tagsubfield for the same reason. Can we agree on this change?
I'd say yes. Patch coming up..
Created attachment 86599 [details] [review] Bug 19670: Change Collation of marc_field to allow mixed case mappings Test plan: 1. Apply patch and update database 2. Verify that you can add a search field mapping for both 100a and 100A. 3. Verify that the above also works with a newly-created database.
Created attachment 87240 [details] [review] Bug 19670: Change Collation of marc_field to allow mixed case mappings Test plan: 1. Apply patch and update database 2. Verify that you can add a search field mapping for both 100a and 100A. 3. Verify that the above also works with a newly-created database. Signed-off-by: Michal Denar <black23@gmail.com>
As this is classified as a bugfix for backporting, I think we should add a check to the database update to make sure it's not run twice (Idempotent https://wiki.koha-community.org/wiki/Database_updates). Could you please check?
I found sometime we did a check, other times not. An example is here: http://git.koha-community.org/gitweb/?p=koha.git;a=blob;f=installer/data/mysql/updatedatabase.pl;h=4f76b76b9b7d00a53b4509291c3409c02af8f201;hb=6666918dd661b0207366becefd1480257197a353#l9712
Created attachment 87379 [details] [review] Bug 19670: Change Collation of marc_field to allow mixed case mappings Test plan: 1. Apply patch and update database 2. Verify that you can add a search field mapping for both 100a and 100A. 3. Verify that the above also works with a newly-created database. Signed-off-by: Michal Denar <black23@gmail.com>
Requesting a new signoff since the check changes the patch so much.
Created attachment 87558 [details] [review] Bug 19670: Change Collation of marc_field to allow mixed case mappings Test plan: 1. Apply patch and update database 2. Verify that you can add a search field mapping for both 100a and 100A. 3. Verify that the above also works with a newly-created database. Signed-off-by: Michal Denar <black23@gmail.com>
Created attachment 87590 [details] [review] Bug 19670: Change Collation of marc_field to allow mixed case mappings Test plan: 1. Apply patch and update database 2. Verify that you can add a search field mapping for both 100a and 100A. 3. Verify that the above also works with a newly-created database. Signed-off-by: Michal Denar <black23@gmail.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Thanks, Ere!
Awesome work all! Pushed to master for 19.05
Pushed to 18.11.x for 18.11.05
backported to 18.05.x for 18.05.12