Bugzilla – Attachment 95770 Details for
Bug 23260
Anonymize (remove) patron data from items_last_borrower
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 23260: Add anonymize items_last_borrower feature
Bug-23260-Add-anonymize-itemslastborrower-feature.patch (text/plain), 17.06 KB, created by
Marcel de Rooy
on 2019-11-25 09:05:22 UTC
(
hide
)
Description:
Bug 23260: Add anonymize items_last_borrower feature
Filename:
MIME Type:
Creator:
Marcel de Rooy
Created:
2019-11-25 09:05:22 UTC
Size:
17.06 KB
patch
obsolete
>From 124ad9c07b9587556e65a55cc4f2fcd6f3c44eac Mon Sep 17 00:00:00 2001 >From: Agustin Moyano <agustinmoyano@theke.io> >Date: Mon, 16 Sep 2019 17:47:43 -0300 >Subject: [PATCH] Bug 23260: Add anonymize items_last_borrower feature >Content-Type: text/plain; charset=utf-8 > >This patch adds the feature to anonymize patrons from items_last_borrower table. This code is run from batch_anonymize script, which is triggered from a cron job. > >To test: >1) apply this patch and the previous one >2) perl installer/data/mysql/updatedatabase.pl >3) set StoreLastBorrower preference to Allow. >4) Create a Check out followed by a Check in. >CHECK => a row should appear in items_last_borrower table with the borrower and the item number. >5) In mysql, update created_on date to one day earlier. >6) set AnonymousPatron preference to a valid patron id >7) perl misc/cronjobs/batch_anonymise.pl >SUCCESS => borrower number in items_last_borrower changed to AnonymousPatron id >8) repeat steps 4 and 5 >9) set AnonymousPatron preference to 0 >10) perl misc/cronjobs/batch_anonymise.pl >SUCCESS => borrower number in items_last_borrower changed to null >11) prove t/db_dependent/Koha/Patrons.t >12) Sign off > >Sponsored-by: Northeast Kansas Library System >Signed-off-by: Maryse Simard <maryse.simard@inlibro.com> > >Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> >--- > Koha/Item/LastPatron.pm | 44 +++++++++++++ > Koha/Item/LastPatrons.pm | 54 +++++++++++++++ > Koha/Patrons.pm | 87 ++++++++++++++++++++---- > misc/cronjobs/batch_anonymise.pl | 5 +- > t/db_dependent/Koha/Patrons.t | 139 ++++++++++++++++++++++++++++++++++++++- > 5 files changed, 315 insertions(+), 14 deletions(-) > create mode 100644 Koha/Item/LastPatron.pm > create mode 100644 Koha/Item/LastPatrons.pm > >diff --git a/Koha/Item/LastPatron.pm b/Koha/Item/LastPatron.pm >new file mode 100644 >index 0000000000..501b34dc1e >--- /dev/null >+++ b/Koha/Item/LastPatron.pm >@@ -0,0 +1,44 @@ >+package Koha::Item::LastPatron; >+ >+# This file is part of Koha. >+# >+# Koha is free software; you can redistribute it and/or modify it under the >+# terms of the GNU General Public License as published by the Free Software >+# Foundation; either version 3 of the License, or (at your option) any later >+# version. >+# >+# Koha is distributed in the hope that it will be useful, but WITHOUT ANY >+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR >+# A PARTICULAR PURPOSE. See the GNU General Public License for more details. >+# >+# You should have received a copy of the GNU General Public License along >+# with Koha; if not, write to the Free Software Foundation, Inc., >+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. >+ >+use Modern::Perl; >+ >+use Carp; >+ >+use Koha::Database; >+ >+use base qw(Koha::Object); >+ >+=head1 NAME >+ >+Koha::Item::LastPatron - Koha Item LastPatron Object class >+ >+=head1 API >+ >+=head2 Class Methods >+ >+=cut >+ >+=head3 type >+ >+=cut >+ >+sub _type { >+ return 'ItemsLastBorrower'; >+} >+ >+1; >diff --git a/Koha/Item/LastPatrons.pm b/Koha/Item/LastPatrons.pm >new file mode 100644 >index 0000000000..3a91b92f94 >--- /dev/null >+++ b/Koha/Item/LastPatrons.pm >@@ -0,0 +1,54 @@ >+package Koha::Item::LastPatrons; >+ >+# This file is part of Koha. >+# >+# Koha is free software; you can redistribute it and/or modify it under the >+# terms of the GNU General Public License as published by the Free Software >+# Foundation; either version 3 of the License, or (at your option) any later >+# version. >+# >+# Koha is distributed in the hope that it will be useful, but WITHOUT ANY >+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR >+# A PARTICULAR PURPOSE. See the GNU General Public License for more details. >+# >+# You should have received a copy of the GNU General Public License along >+# with Koha; if not, write to the Free Software Foundation, Inc., >+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. >+ >+use Modern::Perl; >+ >+use Carp; >+ >+use Koha::Item::LastPatron; >+ >+use Koha::Database; >+ >+use base qw(Koha::Objects); >+ >+=head1 NAME >+ >+Koha::Item::LastPatrons - Koha Item LastPatron Object set class >+ >+=head1 API >+ >+=head2 Class Methods >+ >+=cut >+ >+=head3 type >+ >+=cut >+ >+sub _type { >+ return 'ItemsLastBorrower'; >+} >+ >+=head3 object_class >+ >+=cut >+ >+sub object_class { >+ return 'Koha::Item::LastPatron'; >+} >+ >+1; >diff --git a/Koha/Patrons.pm b/Koha/Patrons.pm >index fbe696ec7a..8f2ffe42f1 100644 >--- a/Koha/Patrons.pm >+++ b/Koha/Patrons.pm >@@ -31,6 +31,7 @@ use Koha::Patron; > use Koha::Exceptions::Patron; > use Koha::Patron::Categories; > use Date::Calc qw( Today Add_Delta_YMD ); >+use Koha::Item::LastPatrons; > > use base qw(Koha::Objects); > >@@ -129,7 +130,7 @@ sub search_upcoming_membership_expires { > > =head3 search_patrons_to_anonymise > >- my $patrons = Koha::Patrons->search_patrons_to_anonymise( { before => $older_than_date, [ library => $library ] } ); >+ my $patrons = Koha::Patrons->search_patrons_to_anonymise( { before => $older_than_date, [ library => $library, for => $table_to_anonymize ] } ); > > This method returns all patrons who has an issue history older than a given date. > >@@ -139,6 +140,7 @@ sub search_patrons_to_anonymise { > my ( $class, $params ) = @_; > my $older_than_date = $params->{before}; > my $library = $params->{library}; >+ my $for = $params->{for}; > $older_than_date = $older_than_date ? dt_from_string($older_than_date) : dt_from_string; > $library ||= > ( C4::Context->preference('IndependentBranches') && C4::Context->userenv && !C4::Context->IsSuperLibrarian() && C4::Context->userenv->{branch} ) >@@ -147,17 +149,35 @@ sub search_patrons_to_anonymise { > my $anonymous_patron = C4::Context->preference('AnonymousPatron') || undef; > > my $dtf = Koha::Database->new->schema->storage->datetime_parser; >- my $rs = $class->_resultset->search( >- { returndate => { '<' => $dtf->format_datetime($older_than_date), }, >- 'old_issues.borrowernumber' => { 'not' => undef }, >- privacy => { '<>' => 0 }, # Keep forever >- ( $library ? ( 'old_issues.branchcode' => $library ) : () ), >- ( $anonymous_patron ? ( 'old_issues.borrowernumber' => { '!=' => $anonymous_patron } ) : () ), >- }, >- { join => ["old_issues"], >- distinct => 1, >- } >- ); >+ my $rs; >+ if($for && $for eq 'items_last_borrower') { >+ $rs = $class->_resultset->search( >+ { 'items_last_borrowers.created_on' => { '<' => $dtf->format_datetime($older_than_date), }, >+ 'items_last_borrowers.borrowernumber' => { 'not' => undef }, >+ 'itemnumber.damaged' => 0, >+ 'itemnumber.itemlost' => 0, >+ 'itemnumber.withdrawn' => 0, >+ ( $library ? ( 'itemnumber.homebranch' => $library ) : () ), >+ ( $anonymous_patron ? ( 'items_last_borrowers.borrowernumber' => { '!=' => $anonymous_patron } ) : () ), >+ }, >+ { join => { "items_last_borrowers" => "itemnumber" }, >+ distinct => 1, >+ } >+ ); >+ } else { >+ #for old_issues >+ $rs = $class->_resultset->search( >+ { returndate => { '<' => $dtf->format_datetime($older_than_date), }, >+ 'old_issues.borrowernumber' => { 'not' => undef }, >+ privacy => { '<>' => 0 }, # Keep forever >+ ( $library ? ( 'old_issues.branchcode' => $library ) : () ), >+ ( $anonymous_patron ? ( 'old_issues.borrowernumber' => { '!=' => $anonymous_patron } ) : () ), >+ }, >+ { join => ["old_issues"], >+ distinct => 1, >+ } >+ ); >+ } > return Koha::Patrons->_new_from_dbic($rs); > } > >@@ -200,6 +220,49 @@ sub anonymise_issue_history { > return $nb_rows; > } > >+=head3 anonymise_items_last_borrower >+ >+ Koha::Patrons->search->anonymise_items_last_borrower( { [ before => $older_than_date ] } ); >+ >+Anonymise last borrower history (items_last_borrower) for all patrons older than the given date (optional). >+To make sure all the conditions are met, the caller has the responsibility to >+call search_patrons_to_anonymise to filter the Koha::Patrons set >+ >+=cut >+ >+sub anonymise_items_last_borrower { >+ my ( $self, $params ) = @_; >+ >+ my $older_than_date = $params->{before}; >+ >+ $older_than_date = dt_from_string $older_than_date if $older_than_date; >+ >+ # The default of 0 does not work due to foreign key constraints >+ # The anonymisation should not fail quietly if AnonymousPatron is not a valid entry >+ # Set it to undef (NULL) >+ my $dtf = Koha::Database->new->schema->storage->datetime_parser; >+ my $nb_rows = 0; >+ while ( my $patron = $self->next ) { >+ my $last_borrowers = Koha::Item::LastPatrons->search( >+ { >+ borrowernumber => $patron->borrowernumber, >+ ( >+ $older_than_date >+ ? ( created_on => >+ { '<' => $dtf->format_datetime($older_than_date) } ) >+ : () >+ ) >+ } >+ ); >+ my $anonymous_patron = C4::Context->preference('AnonymousPatron') || undef; >+ while(my $last_borrower = $last_borrowers->next) { >+ $last_borrower->borrowernumber( $anonymous_patron )->store; >+ $nb_rows++; >+ } >+ } >+ return $nb_rows; >+} >+ > =head3 delete > > Koha::Patrons->search({ some filters here })->delete({ move => 1, verbose => 1 }); >diff --git a/misc/cronjobs/batch_anonymise.pl b/misc/cronjobs/batch_anonymise.pl >index ea63dfa01d..36c05a5068 100755 >--- a/misc/cronjobs/batch_anonymise.pl >+++ b/misc/cronjobs/batch_anonymise.pl >@@ -73,9 +73,12 @@ cronlogaction(); > my ($year,$month,$day) = Today(); > my ($newyear,$newmonth,$newday) = Add_Delta_Days ($year,$month,$day,(-1)*$days); > my $formatdate = sprintf "%4d-%02d-%02d",$newyear,$newmonth,$newday; >-$verbose and print "Checkouts before $formatdate will be anonymised.\n"; >+$verbose and print "Checkouts and items last borrowers before $formatdate will be anonymised.\n"; > > my $rows = Koha::Patrons->search_patrons_to_anonymise( { before => $formatdate } )->anonymise_issue_history( { before => $formatdate } ); > $verbose and print int($rows) . " checkouts anonymised.\n"; > >+$rows = Koha::Patrons->search_patrons_to_anonymise( { before => $formatdate , for => 'items_last_borrower'} )->anonymise_items_last_borrower( { before => $formatdate } ); >+$verbose and print int($rows) . " items last borrower anonymised.\n"; >+ > exit(0); >diff --git a/t/db_dependent/Koha/Patrons.t b/t/db_dependent/Koha/Patrons.t >index 4d7063c497..3e2e76c451 100644 >--- a/t/db_dependent/Koha/Patrons.t >+++ b/t/db_dependent/Koha/Patrons.t >@@ -19,7 +19,7 @@ > > use Modern::Perl; > >-use Test::More tests => 40; >+use Test::More tests => 41; > use Test::Warn; > use Test::Exception; > use Test::MockModule; >@@ -40,6 +40,8 @@ use Koha::Patron::Categories; > use Koha::Patron::Relationship; > use Koha::Database; > use Koha::DateUtils; >+use DateTime; >+use DateTime::Duration; > use Koha::Virtualshelves; > > use t::lib::TestBuilder; >@@ -1178,6 +1180,141 @@ subtest 'search_patrons_to_anonymise & anonymise_issue_history' => sub { > t::lib::Mocks::mock_preference('IndependentBranches', 0); > }; > >+subtest 'search_patrons_to_anonymise & anonymise_items_last_borrower' => sub { >+ plan tests => 3; >+ >+ my $branch = $builder->build({ source => 'Branch' }); >+ my $userenv_patron = $builder->build_object({ >+ class => 'Koha::Patrons', >+ value => { branchcode => $branch->{branchcode}, flags => 0 }, >+ }); >+ t::lib::Mocks::mock_userenv({ patron => $userenv_patron }); >+ t::lib::Mocks::mock_preference('StoreLastBorrower', 1); >+ >+ subtest 'AnonymousPatron is defined' => sub { >+ plan tests => 3; >+ >+ my $anonymous = $builder->build( { source => 'Borrower', }, ); >+ t::lib::Mocks::mock_preference( 'AnonymousPatron', $anonymous->{borrowernumber} ); >+ t::lib::Mocks::mock_preference('IndependentBranches', 0); >+ my $patron = $builder->build( >+ { source => 'Borrower', >+ value => { privacy => 1, } >+ } >+ ); >+ my $item = $builder->build( >+ { source => 'Item', >+ value => { >+ itemlost => 0, >+ withdrawn => 0, >+ damaged => 0 >+ }, >+ } >+ ); >+ my $issue = $builder->build( >+ { source => 'Issue', >+ value => { >+ borrowernumber => $patron->{borrowernumber}, >+ itemnumber => $item->{itemnumber}, >+ }, >+ } >+ ); >+ >+ my $now = DateTime->now; >+ my ( $returned, undef, undef ) = C4::Circulation::AddReturn( $item->{barcode}, undef, undef, $now ); >+ is( $returned, 1, 'The item should have been returned' ); >+ my $before = $now + DateTime::Duration->new( days => 1 ); >+ my $rows_affected = Koha::Patrons->search_patrons_to_anonymise( { before => $before->ymd, for => 'items_last_borrower' } )->anonymise_items_last_borrower( { before => $before->ymd } ); >+ ok( $rows_affected > 0, 'AnonymiseIssueHistory should affect at least 1 row' ); >+ my $dbh = C4::Context->dbh; >+ my ($borrowernumber_used_to_anonymised) = $dbh->selectrow_array(q| >+ SELECT borrowernumber FROM items_last_borrower where itemnumber = ? >+ |, undef, $item->{itemnumber}); >+ is( $borrowernumber_used_to_anonymised, $anonymous->{borrowernumber}, 'AnonymousPatron should have been used to anonymise' ); >+ Koha::Patrons->find( $patron->{borrowernumber})->delete; >+ Koha::Patrons->find( $anonymous->{borrowernumber})->delete; >+ }; >+ >+ subtest 'AnonymousPatron is not defined' => sub { >+ plan tests => 3; >+ >+ t::lib::Mocks::mock_preference('IndependentBranches', 0); >+ t::lib::Mocks::mock_preference( 'AnonymousPatron', '' ); >+ my $patron = $builder->build( >+ { source => 'Borrower', >+ value => { privacy => 1, } >+ } >+ ); >+ my $item = $builder->build( >+ { source => 'Item', >+ value => { >+ itemlost => 0, >+ withdrawn => 0, >+ damaged => 0 >+ }, >+ } >+ ); >+ my $issue = $builder->build( >+ { source => 'Issue', >+ value => { >+ borrowernumber => $patron->{borrowernumber}, >+ itemnumber => $item->{itemnumber}, >+ }, >+ } >+ ); >+ >+ my $now = DateTime->now; >+ my ( $returned, undef, undef ) = C4::Circulation::AddReturn( $item->{barcode}, undef, undef, $now ); >+ is( $returned, 1, 'The item should have been returned' ); >+ my $before = $now + DateTime::Duration->new( days => 1 ); >+ my $rows_affected = Koha::Patrons->search_patrons_to_anonymise( { before => $before->ymd, for => 'items_last_borrower' } )->anonymise_items_last_borrower( { before => $before->ymd } ); >+ ok( $rows_affected > 0, 'AnonymiseIssueHistory should affect at least 1 row' ); >+ my $dbh = C4::Context->dbh; >+ my ($borrowernumber_used_to_anonymised) = $dbh->selectrow_array(q| >+ SELECT borrowernumber FROM items_last_borrower where itemnumber = ? >+ |, undef, $item->{itemnumber}); >+ is( $borrowernumber_used_to_anonymised, undef, 'undef should have been used to anonymise' ); >+ Koha::Patrons->find( $patron->{borrowernumber})->delete; >+ }; >+ >+ subtest 'Logged in librarian is not superlibrarian & IndependentBranches' => sub { >+ plan tests => 1; >+ t::lib::Mocks::mock_preference( 'IndependentBranches', 1 ); >+ my $patron = $builder->build( >+ { source => 'Borrower', >+ value => { privacy => 1 } # Another branchcode than the logged in librarian >+ } >+ ); >+ my $item = $builder->build( >+ { source => 'Item', >+ value => { >+ itemlost => 0, >+ withdrawn => 0, >+ damaged => 0 >+ }, >+ } >+ ); >+ my $issue = $builder->build( >+ { source => 'Issue', >+ value => { >+ borrowernumber => $patron->{borrowernumber}, >+ itemnumber => $item->{itemnumber}, >+ }, >+ } >+ ); >+ >+ my $now = DateTime->now; >+ my ( $returned, undef, undef ) = C4::Circulation::AddReturn( $item->{barcode}, undef, undef, $now ); >+ my $before = $now + DateTime::Duration->new( days => 1 ); >+ is( Koha::Patrons->search_patrons_to_anonymise( { before => $before->ymd, for => 'items_last_borrower' } )->count, 0 ); >+ Koha::Patrons->find( $patron->{borrowernumber})->delete; >+ }; >+ >+ $userenv_patron->delete; >+ t::lib::Mocks::mock_preference('StoreLastBorrower', 0); >+ t::lib::Mocks::mock_preference('IndependentBranches', 0); >+}; >+ > subtest 'libraries_where_can_see_patrons + can_see_patron_infos + search_limited' => sub { > plan tests => 3; > >-- >2.11.0
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 23260
:
92863
|
92864
|
93052
|
94858
|
94859
|
95033
|
95457
|
95458
|
95459
|
95460
|
95651
|
95698
|
95699
|
95700
|
95701
|
95769
|
95770
|
95771
|
95772
|
96791
|
97841
|
97842
|
98044
|
98053
|
98054
|
98064
|
98282
|
98349
|
98352
|
112734
|
112735
|
112736
|
112737
|
112738
|
112739
|
112740
|
112741
|
112742
|
112743
|
117974
|
117975
|
117976
|
117977
|
117978
|
117979
|
117980
|
117981
|
117982
|
117983
|
120522
|
120523
|
120524
|
120525
|
120526
|
120527
|
120528
|
120529
|
120530
|
120531
|
122963
|
122964
|
122965
|
122966
|
122967
|
122968
|
122969
|
122970
|
122971
|
122972
|
145175
|
145176
|
145177
|
145178
|
145179
|
145180
|
145181
|
145182
|
145183
|
145184
|
157106
|
157107
|
157108
|
157109
|
157110
|
157111
|
157112
|
157113
|
157114
|
157115
|
157116
|
159291
|
159292
|
159293
|
159294
|
159298
|
159299
|
159300
|
159301
|
164129
|
164130
|
164131
|
164132
|
164133