|
Description
David Gustafsson
2017-11-28 10:40:47 UTC
Created attachment 69414 [details] [review] Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Add new "merge", "revert" and "add" operations for synchronizing Elasticsearch mappings stored in database with mappings.yaml "add": Only new fields will be added (present in mappings.yaml but not in database), mappings for fields stored in database will be left untouched. "revert": Mappings for all fields in mappings.yaml will be reverted, mappings restored to default onces, and local modifications of those mappings removed. "merge": New mappings for fields present in mappings.yaml will be merged in, keeping any local modifications of marc targets (this could also include old problematic mappings later fixed in mappings.yaml, so should be used with some caution). Just saw that commit message contained no How to test, I should probably fix this and will try to do so in the near future. Hi David, I take a look to your patch to see if I can manage without a test plan but I failed :) I can test it in the next few days if you can provide a test plan ! Created attachment 74819 [details] [review] Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Add new "merge", "revert" and "add" operations for synchronizing Elasticsearch mappings stored in database with mappings.yaml How to test: 1) Make sure mappings.yaml contains mappings not present in database. (Also be aware that steps below will result in changed search engine configuration, so use a test environment and/or make sure to backup database before starting) 2) Go to "Administration" and "Search Engine Configuration" 3) Append ?i_know_what_i_am_doing=1&op=add to url and perform request in browser 4) Vefify that any new mappings in mappings.yaml has been added, but no current mappings has been modified 5) Edit mappings.yaml and add a new mapping for some field (new item below "mappings:" key) 6) Add a new mapping for the same field in the database (through admin ui) 7) Append ?i_know_what_i_am_doing=1&op=merge to url and perform request in browser 8) Verify that the new mapping from mappings.yaml has been added for this field, while the field added in database was preserved 10) Append ?i_know_what_i_am_doing=1&op=revert to url and perform request in browser 11) Vefify that the mapping previously added the field in database has been deleted, and the mappings for this field are identical to those in mappings.yaml Sponsored-by: Gothenburg University Library Created attachment 74820 [details] [review] Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Add new "merge", "revert" and "add" operations for synchronizing Elasticsearch mappings stored in database with mappings.yaml How to test: 1) Make sure mappings.yaml contains mappings not present in database. (Also be aware that steps below will result in changed search engine configuration, so use a test environment and/or make sure to backup database before starting) 2) Go to "Administration" and "Search Engine Configuration" 3) Append ?i_know_what_i_am_doing=1&op=add to url and perform request in browser 4) Verify that any new mappings in mappings.yaml has been added, but no current mappings has been modified or deleted 5) Edit mappings.yaml and add a new mapping for some field (new item below "mappings:" key) 6) Add a new mapping for the same field in the database (through admin ui) 7) Append ?i_know_what_i_am_doing=1&op=merge to url and perform request in browser 8) Verify that the new mapping from mappings.yaml has been added for this field, while the field added in database was preserved 10) Append ?i_know_what_i_am_doing=1&op=revert to url and perform request in browser 11) Verify that the mapping previously added the field in database has been deleted, and the mappings for this field are identical to those in mappings.yaml Sponsored-by: Gothenburg University Library Corrected some spelling mistakes in commit message. @Séverine Queune: thankgs for taking an interest, there now is a test-plan. Hi David, Thank you for your dev, it's a useful one so it's natural to be interested by it :) I can't apply it on master branch (17.12.00.040) : 74820 - Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Using index info to reconstruct a base tree... M Koha/SearchEngine/Elasticsearch.pm Falling back to patching base and 3-way merge... Auto-merging Koha/SearchEngine/Elasticsearch.pm CONFLICT (content): Merge conflict in Koha/SearchEngine/Elasticsearch.pm Failed to merge in the changes. Patch failed at 0001 Bug 19707 - Add new operations for synchronizing Elasticsearch mappings The copy of the patch that failed is found in: /var/repositories/koha/.git/rebase-apply/patch Created attachment 75107 [details] [review] Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Add new "merge", "revert" and "add" operations for synchronizing Elasticsearch mappings stored in database with mappings.yaml Created attachment 75108 [details] [review] Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Add new "merge", "revert" and "add" operations for synchronizing Elasticsearch mappings stored in database with mappings.yaml How to test: 1) Make sure mappings.yaml contains mappings not present in database. (Also be aware that steps below will result in changed search engine configuration, so use a test environment and/or make sure to backup database before starting) 2) Go to "Administration" and "Search Engine Configuration" 3) Append ?i_know_what_i_am_doing=1&op=add to url and perform request in browser 4) Verify that any new mappings in mappings.yaml has been added, but no current mappings has been modified or deleted 5) Edit mappings.yaml and add a new mapping for some field (new item below "mappings:" key) 6) Add a new mapping for the same field in the database (through admin ui) 7) Append ?i_know_what_i_am_doing=1&op=merge to url and perform request in browser 8) Verify that the new mapping from mappings.yaml has been added for this field, while the field added in database was preserved 10) Append ?i_know_what_i_am_doing=1&op=revert to url and perform request in browser 11) Verify that the mapping previously added the field in database has been deleted, and the mappings for this field are identical to those in mappings.yaml Sponsored-by: Gothenburg University Library Rebased against Koha master. Created attachment 79109 [details] [review] Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Add new "merge", "revert" and "add" operations for synchronizing Elasticsearch mappings stored in database with mappings.yaml How to test: 1) Make sure mappings.yaml contains mappings not present in database. (Also be aware that steps below will result in changed search engine configuration, so use a test environment and/or make sure to backup database before starting) 2) Go to "Administration" and "Search Engine Configuration" 3) Append ?i_know_what_i_am_doing=1&op=add to url and perform request in browser 4) Verify that any new mappings in mappings.yaml has been added, but no current mappings has been modified or deleted 5) Edit mappings.yaml and add a new mapping for some field (new item below "mappings:" key) 6) Add a new mapping for the same field in the database (through admin ui) 7) Append ?i_know_what_i_am_doing=1&op=merge to url and perform request in browser 8) Verify that the new mapping from mappings.yaml has been added for this field, while the field added in database was preserved 10) Append ?i_know_what_i_am_doing=1&op=revert to url and perform request in browser 11) Verify that the mapping previously added the field in database has been deleted, and the mappings for this field are identical to those in mappings.yaml Sponsored-by: Gothenburg University Library Something when wrong with the previous rebase, so rebased again and added buttons for all the new operations. Created attachment 79112 [details] [review] Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Add new "merge", "revert" and "add" operations for synchronizing Elasticsearch mappings stored in database with mappings.yaml How to test: 1) Make sure mappings.yaml contains mappings not present in database. (Also be aware that steps below will result in changed search engine configuration, so use a test environment and/or make sure to backup database before starting) 2) Go to "Administration" and "Search Engine Configuration" 3) Click "Add mappings" and confirm 4) Verify that any new mappings in mappings.yaml has been added, but no current mappings has been modified or deleted 5) Edit mappings.yaml and add a new mapping for some field (new item below "mappings:" key) 6) Add a new mapping for the same field in the database (through admin ui) 7) Click "Merge mappings" and confirm 8) Verify that the new mapping from mappings.yaml has been added for this field, while the field added in database was preserved 10) Click "Revert mappings" and confirm 11) Verify that the mapping previously added the field in database has been deleted, and the mappings for this field are identical to those in mappings.yaml Sponsored-by: Gothenburg University Library Fixed commit-message to reflect latest changes. Created attachment 82795 [details] [review] Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Add new "merge", "revert" and "add" operations for synchronizing Elasticsearch mappings stored in database with mappings.yaml How to test: 1) Make sure mappings.yaml contains mappings not present in database. (Also be aware that steps below will result in changed search engine configuration, so use a test environment and/or make sure to backup database before starting) 2) Go to "Administration" and "Search Engine Configuration" 3) Click "Add mappings" and confirm 4) Verify that any new mappings in mappings.yaml has been added, but no current mappings has been modified or deleted 5) Edit mappings.yaml and add a new mapping for some field (new item below "mappings:" key) 6) Add a new mapping for the same field in the database (through admin ui) 7) Click "Merge mappings" and confirm 8) Verify that the new mapping from mappings.yaml has been added for this field, while the field added in database was preserved 10) Click "Revert mappings" and confirm 11) Verify that the mapping previously added the field in database has been deleted, and the mappings for this field are identical to those in mappings.yaml Sponsored-by: Gothenburg University Library Rebase against master. Created attachment 110054 [details] [review] Bug 19707 - Add new operations for synchronizing Elasticsearch mappings Add new "merge", "revert" and "add" operations for synchronizing Elasticsearch mappings stored in database with mappings.yaml How to test: 1) Make sure mappings.yaml contains mappings not present in database. (Also be aware that steps below will result in changed search engine configuration, so use a test environment and/or make sure to backup database before starting) 2) Go to "Administration" and "Search Engine Configuration" 3) Click "Add mappings" and confirm 4) Verify that any new mappings in mappings.yaml has been added, but no current mappings has been modified or deleted 5) Edit mappings.yaml and add a new mapping for some field (new item below "mappings:" key) 6) Add a new mapping for the same field in the database (through admin ui) 7) Click "Merge mappings" and confirm 8) Verify that the new mapping from mappings.yaml has been added for this field, while the field added in database was preserved 10) Click "Revert mappings" and confirm 11) Verify that the mapping previously added the field in database has been deleted, and the mappings for this field are identical to those in mappings.yaml Sponsored-by: Gothenburg University Library Rebased against master. I've got an issue with merging option (steps 6-8) : I can add a new mapping in UI, no problem for saving it but after merging, the field disappear form UI and (obviously) it's not added to yaml file. I didn't succeed to revert the mapping too, but maybe this option is only available combined with the merge option ? Just for my curiosity, how the modifications are stored to enable a revert ? This is such a great feature !!! Can't wait to see it in master ! |