| Summary: | Recent creation of unique index on library_groups erroneously removes rows | ||
|---|---|---|---|
| Product: | Koha | Reporter: | Stefan Berndtsson <stefan.berndtsson> |
| Component: | Architecture, internals, and plumbing | Assignee: | Jonathan Druart <jonathan.druart> |
| Status: | CLOSED FIXED | QA Contact: | Testopia <testopia> |
| Severity: | critical | ||
| Priority: | P2 | CC: | glasklas, jonathan.druart |
| Version: | Main | ||
| Hardware: | All | ||
| OS: | All | ||
| Change sponsored?: | --- | Patch complexity: | --- |
| Documentation contact: | Documentation submission: | ||
| Text to go in the release notes: | Version(s) released in: |
20.05.00
|
|
| Circulation function: | |||
| Bug Depends on: | 21674 | ||
| Bug Blocks: | |||
| Attachments: |
Bug 24741: (bug 21674 follow-up) Fix destructive update DB 19.12.00.017
Bug 24741: (bug 21674 follow-up) Fix destructive update DB 19.12.00.017 Bug 24741: (bug 21674 follow-up) Fix destructive update DB 19.12.00.017 |
||
|
Description
Stefan Berndtsson
2020-02-27 11:46:22 UTC
I'm not 100%, but think a correct query to remove duplicates could be written like: DELETE FROM library_groups WHERE id NOT IN ( SELECT MIN(id) FROM ( SELECT * FROM library_groups ) AS lg GROUP BY parent_id, branchcode ) AND NOT(parent_id IS NULL OR branchcode IS NULL); I tried again. For me it does not remove non-duplicate entries, but it removed an empty subgroup: Before: https://snipboard.io/20aTY9.jpg After : https://snipboard.io/lKzaeE.jpg Subgroup 'yyyy' has been removed (not correct), however 'FPL' has not been removed from the first 'xx' group. Did you notice another behavior? (In reply to David Gustafsson from comment #1) > I'm not 100%, but think a correct query to remove duplicates could be > written like: > > DELETE FROM library_groups > WHERE id NOT IN ( > SELECT MIN(id) > FROM ( SELECT * FROM library_groups ) AS lg > GROUP BY parent_id, branchcode > ) > AND NOT(parent_id IS NULL OR branchcode IS NULL); This query looks correct to me. Could you submit a patch? (In reply to Jonathan Druart from comment #2) > I tried again. > For me it does not remove non-duplicate entries, but it removed an empty > subgroup: > > Before: https://snipboard.io/20aTY9.jpg > After : https://snipboard.io/lKzaeE.jpg > > Subgroup 'yyyy' has been removed (not correct), however 'FPL' has not been > removed from the first 'xx' group. > > Did you notice another behavior? Yes. In our case we had: https://snipboard.io/cLhEY7.jpg Since I've restored the data, I don't have an after-image, but after the migration ACQ was the only one left under __SEARCH_GROUPS__. All the others had been removed, with their entire substructure. ACQ is the one with the lowest "id", and MIN(id) therefor kept that one. Created attachment 99978 [details] [review] Bug 24741: (bug 21674 follow-up) Fix destructive update DB 19.12.00.017 Previous version of the SQL query did not deal correctly with subgroup. Test plan: checkout the following commit to be just before 017 commit 715da06db557edc4b1baa51cc278fdac362c01c4 Bug 22868: DBRev 19.12.00.016 reset_all # recreate the DB Create a complex tree of library groups, with several groups, subgroups, etc. When you add a new library, refresh the page to add it several time. When ready, git checkout master, restart all the things, execute the updatedatabase script, then go to the library groups page. You should see that the duplicated entries have been removed, but subgroups have not been deleted Created attachment 100041 [details] [review] Bug 24741: (bug 21674 follow-up) Fix destructive update DB 19.12.00.017 Previous version of the SQL query did not deal correctly with subgroup. Test plan: checkout the following commit to be just before 017 commit 715da06db557edc4b1baa51cc278fdac362c01c4 Bug 22868: DBRev 19.12.00.016 reset_all # recreate the DB Create a complex tree of library groups, with several groups, subgroups, etc. When you add a new library, refresh the page to add it several time. When ready, git checkout master, restart all the things, execute the updatedatabase script, then go to the library groups page. You should see that the duplicated entries have been removed, but subgroups have not been deleted Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Tested with duplicated entries on groups and subgroups, going back to the right commit and doing clean install. Patch works, subgroups are preserved. No errors. Created attachment 100094 [details] [review] Bug 24741: (bug 21674 follow-up) Fix destructive update DB 19.12.00.017 Previous version of the SQL query did not deal correctly with subgroup. Test plan: checkout the following commit to be just before 017 commit 715da06db557edc4b1baa51cc278fdac362c01c4 Bug 22868: DBRev 19.12.00.016 reset_all # recreate the DB Create a complex tree of library groups, with several groups, subgroups, etc. When you add a new library, refresh the page to add it several time. When ready, git checkout master, restart all the things, execute the updatedatabase script, then go to the library groups page. You should see that the duplicated entries have been removed, but subgroups have not been deleted Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Tested with duplicated entries on groups and subgroups, going back to the right commit and doing clean install. Patch works, subgroups are preserved. No errors. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Nice work everyone! Pushed to master for 20.05 |
