27883 – Add ability to preserve patron field from being overwritten by import

Bug 27883 - Add ability to preserve patron field from being overwritten by import

Summary: Add ability to preserve patron field from being overwritten by import

Status:	CLOSED FIXED

Alias:	None

Product:	Koha
Classification:	Unclassified
Component:	Tools (show other bugs)
Version:	Main
Hardware:	All All

Importance:	P5 - low enhancement
Assignee:	Kyle M Hall (khall)
QA Contact:	Testopia

URL:
Keywords:	Manual

Duplicates (1):	24115 (view as bug list)
Depends on:
Blocks:	27920
	Show dependency tree / graph

Reported:	2021-03-05 15:18 UTC by Kyle M Hall (khall)
Modified:	2022-12-12 21:24 UTC (History)
CC List:	5 users (show)

See Also:
Change sponsored?:	---
Patch complexity:	Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:	This enhancement to the patron import tool lets you keep current values for selected fields for existing patrons - when the data is imported the selected fields are not overwritten. When importing: - match to existing patrons using either their card number or user name - select the fields that will not be overwritten under 'Preserve existing values'.
Version(s) released in:	21.11.00
Circulation function:

Attachments
Bug 27883: Add ability to preserve patron field from being overwritten by import (12.49 KB, patch) 2021-03-05 15:24 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: (follow-up) Make label match input id (1.85 KB, patch) 2021-03-30 15:09 UTC, Owen Leonard	Details \| Diff \| Splinter Review
Bug 27883: Add ability to preserve patron field from being overwritten by import (12.49 KB, patch) 2021-04-06 15:52 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: (follow-up) Make label match input id (1.86 KB, patch) 2021-04-06 15:52 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: (QA follow-up) Fix missing variable rename (691 bytes, patch) 2021-04-06 15:52 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: Add ability to preserve patron field from being overwritten by import (12.49 KB, patch) 2021-04-06 15:54 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: (follow-up) Make label match input id (1.92 KB, patch) 2021-04-06 15:54 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: Add ability to preserve patron field from being overwritten by import (12.49 KB, patch) 2021-04-09 09:58 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: (follow-up) Make label match input id (1.92 KB, patch) 2021-04-09 09:58 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: Add ability to preserve patron field from being overwritten by import (12.50 KB, patch) 2021-04-09 11:22 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: (follow-up) Make label match input id (1.92 KB, patch) 2021-04-09 11:22 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: Add ability to preserve patron field from being overwritten by import (12.57 KB, patch) 2021-04-10 22:01 UTC, David Nind	Details \| Diff \| Splinter Review
Bug 27883: (follow-up) Make label match input id (1.95 KB, patch) 2021-04-10 22:01 UTC, David Nind	Details \| Diff \| Splinter Review
Bug 27883: Add ability to preserve patron field from being overwritten by import (12.30 KB, patch) 2021-06-12 10:51 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: (follow-up) Make label match input id (1.96 KB, patch) 2021-06-12 10:51 UTC, Kyle M Hall (khall)	Details \| Diff \| Splinter Review
Bug 27883: Add ability to preserve patron field from being overwritten by import (12.35 KB, patch) 2021-07-09 11:51 UTC, Martin Renvoize (ashimema)	Details \| Diff \| Splinter Review
Bug 27883: (follow-up) Make label match input id (2.01 KB, patch) 2021-07-09 11:51 UTC, Martin Renvoize (ashimema)	Details \| Diff \| Splinter Review
Show Obsolete (15) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description Kyle M Hall (khall) 2021-03-05 15:18:58 UTC

Some libraries would like to be able to preserve particular fields for existing patrons when overwriting them via the patron import tool. Effectively, this means the specified columns of the CSV are used for new patrons, but ignored for existing patrons.

Comment 1 Kyle M Hall (khall) 2021-03-05 15:24:27 UTC Comment hidden (obsolete)

Created attachment 117861 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Comment 2 Owen Leonard 2021-03-30 15:00:40 UTC

Sorry, I can't get this to work! The interface says that my CSV has been imported successfully and that my patron was overwritten, but the expected changes are not made to the existing record.

Comment 3 Owen Leonard 2021-03-30 15:09:19 UTC

The QA tool shows this failure:

 FAIL	misc/import_patrons.pl
   FAIL	  critic
		# Variables::ProhibitUnusedVariables: Got 1 violation(s).
   FAIL	  valid
		Global symbol "@preserve_fields" requires explicit package name (did you forget to declare "my @preserve_fields"?)
		Global symbol "@preserve_fields" requires explicit package name (did you forget to declare "my @preserve_fields"?)
		misc/import_patrons.pl had compilation errors.

Comment 4 Owen Leonard 2021-03-30 15:09:57 UTC Comment hidden (obsolete)

Created attachment 119018 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Comment 5 Kyle M Hall (khall) 2021-04-06 15:52:19 UTC Comment hidden (obsolete)

Created attachment 119227 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Comment 6 Kyle M Hall (khall) 2021-04-06 15:52:31 UTC Comment hidden (obsolete)

Created attachment 119228 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Comment 7 Kyle M Hall (khall) 2021-04-06 15:52:33 UTC Comment hidden (obsolete)

Created attachment 119229 [details] [review]
Bug 27883: (QA follow-up) Fix missing variable rename

Comment 8 Kyle M Hall (khall) 2021-04-06 15:54:34 UTC Comment hidden (obsolete)

Created attachment 119230 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Comment 9 Kyle M Hall (khall) 2021-04-06 15:54:47 UTC Comment hidden (obsolete)

Created attachment 119231 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Comment 10 Kyle M Hall (khall) 2021-04-09 09:58:23 UTC Comment hidden (obsolete)

Created attachment 119366 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Comment 11 Kyle M Hall (khall) 2021-04-09 09:58:33 UTC Comment hidden (obsolete)

Created attachment 119367 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Comment 12 Kyle M Hall (khall) 2021-04-09 11:22:46 UTC Comment hidden (obsolete)

Created attachment 119370 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Comment 13 Kyle M Hall (khall) 2021-04-09 11:22:57 UTC Comment hidden (obsolete)

Created attachment 119371 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Comment 14 David Nind 2021-04-10 22:01:00 UTC

Created attachment 119436 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Signed-off-by: David Nind <david@davidnind.com>

Comment 15 David Nind 2021-04-10 22:01:04 UTC

Created attachment 119437 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: David Nind <david@davidnind.com>

Comment 16 Kyle M Hall (khall) 2021-06-12 10:51:45 UTC

Created attachment 121883 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Signed-off-by: David Nind <david@davidnind.com>

Comment 17 Kyle M Hall (khall) 2021-06-12 10:51:57 UTC

Created attachment 121884 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: David Nind <david@davidnind.com>

Comment 18 Martin Renvoize (ashimema) 2021-07-09 11:51:15 UTC

Created attachment 122719 [details] [review]
Bug 27883: Add ability to preserve patron field from being overwritten by import

Some libraries would like to be able to preserve particular fields for
existing patrons when overwriting them via the patron import tool.
Effectively, this means the specified columns of the CSV are used for
new patrons, but ignored for existing patrons.

Test Plan:
1) Create a patron CSV with one new patron, make the surname and
   firstname "Test1". Add a cardnumber so we can upload it again later.
2) Import the file
3) Change the firstname and surname in the CSV to "Test2"
4) Return to the patron import tool, choose to match on cardnumber,
   overwrite existing patrons, and preserve exiting firstnames
5) Import the file with these settings
6) Referesh the patron details for this patron, the patron's surname
   should still be "Test" while the firstname should now be "Test2"

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Comment 19 Martin Renvoize (ashimema) 2021-07-09 11:51:21 UTC

Created attachment 122720 [details] [review]
Bug 27883: (follow-up) Make label match input id

This change alters the list of columns under "Preserve existing values,"
changing the label for each checkbox can be clicked to trigger the
checkbox change.

To test, apply the patch and refresh the patron import page. Expand the
"Preserve existing values" section and click any label in the list. The
corresponding checkbox should be checked.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Comment 20 Martin Renvoize (ashimema) 2021-07-09 11:52:19 UTC

Code checks out ok, feature works as expected. Tests included and passing,QA script is happy.

Passing QA

Comment 21 Jonathan Druart 2021-07-12 13:48:56 UTC

Pushed to master for 21.11, thanks to everybody involved!

Comment 22 Kyle M Hall (khall) 2021-07-16 12:43:25 UTC

Enhancement not backported to 21.05

Comment 23 Katrin Fischer 2021-07-19 10:38:11 UTC

(In reply to Kyle M Hall from comment #22)
> Enhancement not backported to 21.05

There is something odd going on here. Strings from this have showed up in 21.05 translations I did today?

Comment 24 Katrin Fischer 2021-07-19 10:39:39 UTC

https://translate.koha-community.org/de/21.05/translate/#search=Preserve%20existing%20values&sfields=source,target

Comment 25 Katrin Fischer 2021-07-19 20:54:41 UTC

(In reply to Katrin Fischer from comment #24)
> https://translate.koha-community.org/de/21.05/translate/
> #search=Preserve%20existing%20values&sfields=source,target

Turned out to be a mistake on creating the po files - Koha codebase is ok and po files are now too :)

Comment 26 Jonathan Druart 2021-12-14 14:13:31 UTC

This is causing the following warning:
[2021/12/14 14:12:13] [WARN] CGI::param called in list context from /kohadevbox/koha/tools/import_borrowers.pl line 94, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 412.

Comment 27 Katrin Fischer 2022-01-02 15:22:32 UTC

*** Bug 24115 has been marked as a duplicate of this bug. ***