Bug 27883

Summary: Add ability to preserve patron field from being overwritten by import
Product: Koha Reporter: Kyle M Hall <kyle>
Component: ToolsAssignee: Kyle M Hall <kyle>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: david, jeremy.evans, jonathan.druart, lucy.vaux-harvey, martin.renvoize
Version: MainKeywords: Manual
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
This enhancement to the patron import tool lets you keep current values for selected fields for existing patrons - when the data is imported the selected fields are not overwritten. When importing: - match to existing patrons using either their card number or user name - select the fields that will not be overwritten under 'Preserve existing values'.
Version(s) released in:
21.11.00
Bug Depends on:    
Bug Blocks: 27920    
Attachments: Bug 27883: Add ability to preserve patron field from being overwritten by import
Bug 27883: (follow-up) Make label match input id
Bug 27883: Add ability to preserve patron field from being overwritten by import
Bug 27883: (follow-up) Make label match input id
Bug 27883: (QA follow-up) Fix missing variable rename
Bug 27883: Add ability to preserve patron field from being overwritten by import
Bug 27883: (follow-up) Make label match input id
Bug 27883: Add ability to preserve patron field from being overwritten by import
Bug 27883: (follow-up) Make label match input id
Bug 27883: Add ability to preserve patron field from being overwritten by import
Bug 27883: (follow-up) Make label match input id
Bug 27883: Add ability to preserve patron field from being overwritten by import
Bug 27883: (follow-up) Make label match input id
Bug 27883: Add ability to preserve patron field from being overwritten by import
Bug 27883: (follow-up) Make label match input id
Bug 27883: Add ability to preserve patron field from being overwritten by import
Bug 27883: (follow-up) Make label match input id

Description Kyle M Hall 2021-03-05 15:18:58 UTC
Some libraries would like to be able to preserve particular fields for existing patrons when overwriting them via the patron import tool. Effectively, this means the specified columns of the CSV are used for new patrons, but ignored for existing patrons.
Comment 1 Kyle M Hall 2021-03-05 15:24:27 UTC Comment hidden (obsolete)
Comment 2 Owen Leonard 2021-03-30 15:00:40 UTC
Sorry, I can't get this to work! The interface says that my CSV has been imported successfully and that my patron was overwritten, but the expected changes are not made to the existing record.
Comment 3 Owen Leonard 2021-03-30 15:09:19 UTC
The QA tool shows this failure:

 FAIL	misc/import_patrons.pl
   FAIL	  critic
		# Variables::ProhibitUnusedVariables: Got 1 violation(s).
   FAIL	  valid
		Global symbol "@preserve_fields" requires explicit package name (did you forget to declare "my @preserve_fields"?)
		Global symbol "@preserve_fields" requires explicit package name (did you forget to declare "my @preserve_fields"?)
		misc/import_patrons.pl had compilation errors.
Comment 4 Owen Leonard 2021-03-30 15:09:57 UTC Comment hidden (obsolete)
Comment 5 Kyle M Hall 2021-04-06 15:52:19 UTC Comment hidden (obsolete)
Comment 6 Kyle M Hall 2021-04-06 15:52:31 UTC Comment hidden (obsolete)
Comment 7 Kyle M Hall 2021-04-06 15:52:33 UTC Comment hidden (obsolete)
Comment 8 Kyle M Hall 2021-04-06 15:54:34 UTC Comment hidden (obsolete)
Comment 9 Kyle M Hall 2021-04-06 15:54:47 UTC Comment hidden (obsolete)
Comment 10 Kyle M Hall 2021-04-09 09:58:23 UTC Comment hidden (obsolete)
Comment 11 Kyle M Hall 2021-04-09 09:58:33 UTC Comment hidden (obsolete)
Comment 12 Kyle M Hall 2021-04-09 11:22:46 UTC Comment hidden (obsolete)
Comment 13 Kyle M Hall 2021-04-09 11:22:57 UTC Comment hidden (obsolete)
Comment 14 David Nind 2021-04-10 22:01:00 UTC
Created attachment 119436 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Signed-off-by: David Nind <david@davidnind.com>
Comment 15 David Nind 2021-04-10 22:01:04 UTC
Created attachment 119437 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: David Nind <david@davidnind.com>
Comment 16 Kyle M Hall 2021-06-12 10:51:45 UTC
Created attachment 121883 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Signed-off-by: David Nind <david@davidnind.com>
Comment 17 Kyle M Hall 2021-06-12 10:51:57 UTC
Created attachment 121884 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: David Nind <david@davidnind.com>
Comment 18 Martin Renvoize 2021-07-09 11:51:15 UTC
Created attachment 122719 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 19 Martin Renvoize 2021-07-09 11:51:21 UTC
Created attachment 122720 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 20 Martin Renvoize 2021-07-09 11:52:19 UTC
Code checks out ok, feature works as expected. Tests included and passing,QA script is happy.

Passing QA
Comment 21 Jonathan Druart 2021-07-12 13:48:56 UTC
Pushed to master for 21.11, thanks to everybody involved!
Comment 22 Kyle M Hall 2021-07-16 12:43:25 UTC
Enhancement not backported to 21.05
Comment 23 Katrin Fischer 2021-07-19 10:38:11 UTC
(In reply to Kyle M Hall from comment #22)
> Enhancement not backported to 21.05

There is something odd going on here. Strings from this have showed up in 21.05 translations I did today?
Comment 25 Katrin Fischer 2021-07-19 20:54:41 UTC
(In reply to Katrin Fischer from comment #24)
> https://translate.koha-community.org/de/21.05/translate/
> #search=Preserve%20existing%20values&sfields=source,target

Turned out to be a mistake on creating the po files - Koha codebase is ok and po files are now too :)
Comment 26 Jonathan Druart 2021-12-14 14:13:31 UTC
This is causing the following warning:
[2021/12/14 14:12:13] [WARN] CGI::param called in list context from /kohadevbox/koha/tools/import_borrowers.pl line 94, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 412.
Comment 27 Katrin Fischer 2022-01-02 15:22:32 UTC
*** Bug 24115 has been marked as a duplicate of this bug. ***