Bug 25707 - Mappings update in bug 11529 causes incorrect MARC to DB data flow
Summary: Mappings update in bug 11529 causes incorrect MARC to DB data flow
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: master
Hardware: All All
: P5 - low critical (vote)
Assignee: Kyle M Hall
QA Contact: Tomás Cohen Arazi
URL:
Keywords:
Depends on: 11529
Blocks:
  Show dependency treegraph
 
Reported: 2020-06-10 10:05 UTC by Nick Clemens
Modified: 2020-11-30 21:51 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Text to go in the release notes:
Version(s) released in:
20.11.00, 20.05.01, 19.11.07


Attachments
Bug 25707: (bug 11529 follow-up) Do not update marc mappings, print old mappings to upgrade log (2.04 KB, patch)
2020-06-10 10:08 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 25707: (bug 11529 follow-up) Do not update marc mappings, print old mappings to upgrade log (2.10 KB, patch)
2020-06-11 18:53 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 25707: (QA follow-up) Clearer message about what's going on (1.86 KB, patch)
2020-06-11 18:53 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens 2020-06-10 10:05:13 UTC
Turns out the database update for this change causes unexpected changes to the "Koha to MARC mapping" (via the MARC frameworks)
Comment 1 Nick Clemens 2020-06-10 10:08:39 UTC
Created attachment 105680 [details] [review]
Bug 25707: (bug 11529 follow-up)  Do not update marc mappings, print old mappings to upgrade log

This patch removes the update of marc frameworks from the updatedatabase script and warns the
"MARC to Keyword" (fieldmappings ) to the upgrade log so they can be replaced manually

Signed-off-by: David Cook <dcook@prosentient.com.au>
Comment 2 David Cook 2020-06-11 00:06:46 UTC
I wouldn't say it causes data loss because the data is still in the MARC record but it causes... incorrect MARC to DB mappings?
Comment 3 Nick Clemens 2020-06-11 16:35:21 UTC
(In reply to David Cook from comment #2)
> I wouldn't say it causes data loss because the data is still in the MARC
> record but it causes... incorrect MARC to DB mappings?

It overwrites existing mappings and they are not preserved, so the only way to recover is from backup.
Comment 4 Tomás Cohen Arazi 2020-06-11 18:13:08 UTC
@Kyle I don't understand why are you taking out the WHERE condition. And the warning message could be more clear about why they are wrong, otherwise it just says 'go remap' to what?.
Comment 5 Nick Clemens 2020-06-11 18:22:33 UTC
The where condition was there to not warn the mappings that we attempted to update. We are no longer making this attempt so we want to list the whole table

Something like:
The keyword to marc mapping feature is no longer supported. Above find the mappings that had been defined in your system. You will need to remap any desired MARC fields to the Koha field you desire in the Koha to MARC mappings page under Administration
Comment 6 Tomás Cohen Arazi 2020-06-11 18:23:50 UTC
(In reply to Nick Clemens from comment #5)
> The where condition was there to not warn the mappings that we attempted to
> update. We are no longer making this attempt so we want to list the whole
> table

I get it.

> Something like:
> The keyword to marc mapping feature is no longer supported. Above find the
> mappings that had been defined in your system. You will need to remap any
> desired MARC fields to the Koha field you desire in the Koha to MARC
> mappings page under Administration

That reads better :-D
Comment 7 Tomás Cohen Arazi 2020-06-11 18:53:42 UTC
Created attachment 105771 [details] [review]
Bug 25707: (bug 11529 follow-up) Do not update marc mappings, print old mappings to upgrade log

This patch removes the update of marc frameworks from the updatedatabase script and warns the
"MARC to Keyword" (fieldmappings ) to the upgrade log so they can be replaced manually

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 8 Tomás Cohen Arazi 2020-06-11 18:53:48 UTC
Created attachment 105772 [details] [review]
Bug 25707: (QA follow-up) Clearer message about what's going on

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 9 David Cook 2020-06-12 04:50:14 UTC
(In reply to Nick Clemens from comment #3)
> (In reply to David Cook from comment #2)
> > I wouldn't say it causes data loss because the data is still in the MARC
> > record but it causes... incorrect MARC to DB mappings?
> 
> It overwrites existing mappings and they are not preserved, so the only way
> to recover is from backup.

Ohh I see what you mean. I thought you meant MARC data would be lost, but you're referring to the mapping data. That makes sense. 

Honestly, I didn't recover from the backups. I just reset the mappings to reasonable defaults.
Comment 10 David Cook 2020-06-12 04:51:14 UTC
(In reply to David Cook from comment #9)
> (In reply to Nick Clemens from comment #3)
> > (In reply to David Cook from comment #2)
> > > I wouldn't say it causes data loss because the data is still in the MARC
> > > record but it causes... incorrect MARC to DB mappings?
> > 
> > It overwrites existing mappings and they are not preserved, so the only way
> > to recover is from backup.
> 
> Ohh I see what you mean. I thought you meant MARC data would be lost, but
> you're referring to the mapping data. That makes sense. 
> 
> Honestly, I didn't recover from the backups. I just reset the mappings to
> reasonable defaults.

Since the mappings can be re-created, I wasn't thinking of it as data loss, since the valuable data (the MARC data) is still preserved.
Comment 11 Jonathan Druart 2020-06-12 12:18:00 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 12 Lucas Gass 2020-06-16 22:09:30 UTC
backported to 20.05.x for 20.05.01
Comment 13 Aleisha Amohia 2020-06-17 00:03:39 UTC
backported to 19.11.x for 19.11.07
Comment 14 Victor Grousset/tuxayo 2020-06-18 15:25:07 UTC
Missing dependencies for 19.05.x, no backport. 19.05.x shouldn't be affected.