Bug 23295 - Automatically debar patrons if SMS or email notice fail
Summary: Automatically debar patrons if SMS or email notice fail
Status: Needs Signoff
Alias: None
Product: Koha
Classification: Unclassified
Component: Notices (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Aleisha Amohia
QA Contact: Testopia
URL:
Keywords: no-sandbox
Depends on: 31095
Blocks:
  Show dependency treegraph
 
Reported: 2019-07-10 22:30 UTC by Alex Buckley
Modified: 2024-01-23 14:36 UTC (History)
8 users (show)

See Also:
Change sponsored?: Sponsored
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail (19.61 KB, patch)
2019-07-12 00:01 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail (19.48 KB, patch)
2020-03-30 02:42 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail (19.50 KB, patch)
2020-03-31 02:13 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail (20.25 KB, patch)
2020-05-18 06:03 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail (20.36 KB, patch)
2021-02-01 20:56 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail (20.17 KB, patch)
2023-11-09 01:09 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail (20.10 KB, patch)
2024-01-10 21:18 UTC, Aleisha Amohia
Details | Diff | Splinter Review
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail (20.87 KB, patch)
2024-01-11 19:07 UTC, Aleisha Amohia
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Buckley 2019-07-10 22:30:33 UTC
When the sending of an SMS (or email) notice fails at the Koha level (i.e. before it is passed to the SMS API) then librarians should have the ability to enable automatic restriction (debarment) of patrons prompting them to update their smsalertnumber (or email address) in their library accounts. 

Adding this ability will tie in nicely with the functionality implemented in bug 13552 and bug 23291.

As the enhancement for this bug report would let us restrict when notices failed at the Koha level, whereas 13552 and 23291 would let librarians restrict when notices failed at the SMS API level in batch restriction jobs retrieving patrons by SMS numbers in the SMS gateway report libraries get telling them which notices failed at the SMS API level. 

So all three enhancements together will mean there will be end-to-end functionality to restrict patrons when SMS notices fail, thereby prompting them to update their smsalertnumber.
Comment 1 Alex Buckley 2019-07-12 00:01:49 UTC
Created attachment 91471 [details] [review]
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail

When the 'RestrictPatronsWithFailedNotices' syspref is enabled then
patrons with email and sms notices which failed sending (have a
message_queue.status field of 'failed') have a restriction (debarment)
applied to them.

Test plan:
1. In the Koha staff client > Tools > Overdue notice/status triggers and
create the 'First' rule for all patron categories as:

Delay: 1
Letter: Overdue Notice
SMS: ticked

2. In the system preferences make sure you enter dummy data into the
SMSSendUsername, SMSSendPassword and EmailSMSSendDriverFromAddress
sysprefs

2. Find two non-debarred patrons and make sure they have invalid SMS numbers set

3. Check one item out to each patron in step 2

4. Jump into the database and run the query:

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower1>;

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower2>;

5. Go to  misc/cronjobs directory and enter the Koha shell:

sudo koha-shell <instancename>

6. Run: ./overdue_notices.pl

7. Exit the shell and jump back into the database and run the query:

SELECT message_transport_type, status FROM message_queue WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

8. Confirm both new notice records have the message_transport_type is 'sms' and the status of 'pending'

9. Exit the database and re-enter the Koha shell and run the command:
./process_message_queue.pl

10. Jump back into the database re-run the query from step 7 and confirm
the status is 'failed' for both

11. Also run the query:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice there is no added debarment to these two patrons

12. Apply patch, restart memcached and plack. In the
installer/data/mysql directory enter the Koha shell and run the command:
./update_database.pl

13. In the Administration > Global System Preferences interface of the
staff client notice there is a new system (set to "Don't" by default)
named 'RestrictPatronsWithFailedNotices'.

Enable it (i.e. select 'Do')

14. Create a new file in the /etc/cron.daily directory named koha-custom and
add the following line to it:

koha-foreach --chdir --enabled
/usr/share/koha/bin/cronjobs/restrict_patrons_with_failed_notices.pl

15. In the  misc/cronjobs directory enter the Koha shell and run the command:
./restrict_patrons_with_failed_notices.pl

16. The script should output text saying: There are borrowers with
failed SMS or email notices

However because you haven't given the script the argument -c it won't
apply debarments (restrictions) to any of the patrons with the failed
SMS or email notices.

16. Query the borrower_debarments table:

SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice they still have no restriction

17. Now in the Koha shell run the command:
./restrict_patrons_with_failed_notices.pl -c

18. Notice the script outputs the text:
There are borrowers with failed SMS or email notices
Applying restriction to patron <borrowernumber>: <borrower firstname>
<borrower surname>;

19. Repeat step 16 and notice both patrons now have 1 restriction each
with the borrower_debarments.type=SUSPENSION and comment=SMSnumber
invalid and expiration=NULL

20. Query the borrowers table:
SELECT debarred, debarredcomment FROM borrowers WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

21. Notice the values are:
debarred= 9999-12-31
debarredcomment= SMS number invalid

22. Repeat step 17 and notice the script outputs:
There are borrowers with failed SMS or email notices

Patron <borrowernumber>: <borrower firstname> <borrower surname> is currently restricted due to having an invalid SMS number. No new restriction applied"

23. Repeat step 16 and notice no new debarment has been added to those
borrowers as they have already been restricted from having a failed SMS
notice.

24. In the Koha home directory run the command:
prove t/db_dependent/Koha/Notices.t

This unit test contains the tests for the new subroutines added to
Koha/Notice/Message.pm which are restrict_patron_when_notice_fails() and
get_failed_notices()

25. All tests should pass

26. Sign off

Sponsored-By: Brimbank Library, Australia
Comment 2 Hugo Agud 2020-02-14 10:33:04 UTC
warning: 3 lines applied after fixing whitespace errors.
Falling back to patching base and 3-way merge...
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/patrons.pref
Auto-merging installer/data/mysql/sysprefs.sql
Auto-merging C4/Letters.pm
CONFLICT (content): Merge conflict in C4/Letters.pm
error: Failed to merge in the changes.
Patch failed at 0001 Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-23295-Automatically-restrict-debar-patrons-whe-6bIBRK.patch
Comment 3 Aleisha Amohia 2020-03-30 02:42:36 UTC
Created attachment 102068 [details] [review]
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail

When the 'RestrictPatronsWithFailedNotices' syspref is enabled then
patrons with email and sms notices which failed sending (have a
message_queue.status field of 'failed') have a restriction (debarment)
applied to them.

Test plan:
1. In the Koha staff client > Tools > Overdue notice/status triggers and
create the 'First' rule for all patron categories as:

Delay: 1
Letter: Overdue Notice
SMS: ticked

Ensure you have an SMS notice for the ODUE letter.

2. In the system preferences make sure you enter dummy data into the
SMSSendUsername, SMSSendPassword and EmailSMSSendDriverFromAddress
sysprefs

2. Find two non-debarred patrons and make sure they have invalid SMS numbers set. The SMS numbers must be INCORRECT, for example "123" as an SMS number. Leaving this field empty will result in the message_transport_type defaulting to 'print' instead of 'sms'.

3. Check one item out to each patron in step 2

4. Jump into the database and run the query:

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower1>;

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower2>;

5. Go to  misc/cronjobs directory and enter the Koha shell:

sudo koha-shell <instancename>

6. Run: ./overdue_notices.pl

7. Exit the shell and jump back into the database and run the query:

SELECT message_transport_type, status FROM message_queue WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

8. Confirm both new notice records have the message_transport_type is 'sms' and the status of 'pending'

9. Exit the database and re-enter the Koha shell and run the command:
./process_message_queue.pl

10. Jump back into the database re-run the query from step 7 and confirm
the status is 'failed' for both

11. Also run the query:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice there is no added debarment to these two patrons

12. Apply patch, restart memcached and plack. In the
installer/data/mysql directory enter the Koha shell and run the command:
./update_database.pl

13. In the Administration > Global System Preferences interface of the
staff client notice there is a new system (set to "Don't" by default)
named 'RestrictPatronsWithFailedNotices'.

Enable it (i.e. select 'Do')

14. Create a new file in the /etc/cron.daily directory named koha-custom and
add the following line to it:

koha-foreach --chdir --enabled
/usr/share/koha/bin/cronjobs/restrict_patrons_with_failed_notices.pl

15. In the  misc/cronjobs directory enter the Koha shell and run the command:
./restrict_patrons_with_failed_notices.pl

16. The script should output text saying: There are borrowers with
failed SMS or email notices

However because you haven't given the script the argument -c it won't
apply debarments (restrictions) to any of the patrons with the failed
SMS or email notices.

16. Query the borrower_debarments table:

SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice they still have no restriction

17. Now in the Koha shell run the command:
./restrict_patrons_with_failed_notices.pl -c

18. Notice the script outputs the text:
There are borrowers with failed SMS or email notices
Applying restriction to patron <borrowernumber>: <borrower firstname>
<borrower surname>;

19. Repeat step 16 and notice both patrons now have 1 restriction each
with the borrower_debarments.type=SUSPENSION and comment=SMSnumber
invalid and expiration=NULL

20. Query the borrowers table:
SELECT debarred, debarredcomment FROM borrowers WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

21. Notice the values are:
debarred= 9999-12-31
debarredcomment= SMS number invalid

22. Repeat step 17 and notice the script outputs:
There are borrowers with failed SMS or email notices

Patron <borrowernumber>: <borrower firstname> <borrower surname> is currently restricted due to having an invalid SMS number. No new restriction applied"

23. Repeat step 16 and notice no new debarment has been added to those
borrowers as they have already been restricted from having a failed SMS
notice.

24. In the Koha home directory run the command:
prove t/db_dependent/Koha/Notices.t

This unit test contains the tests for the new subroutines added to
Koha/Notice/Message.pm which are restrict_patron_when_notice_fails() and
get_failed_notices()

25. All tests should pass

26. Sign off

Sponsored-By: Brimbank Library, Australia
Comment 4 Aleisha Amohia 2020-03-30 02:43:57 UTC
Rebased, however tests fail:

$ prove -v t/db_dependent/Koha/Notices.t 
t/db_dependent/Koha/Notices.t .. 
1..8
ok 1 - The template should have been added
ok 2 - Find a notice template by pk should return the correct template
ok 3 - Delete should have deleted the template
ok 4 - No failed notices currently exist
not ok 5 - One failed notice currently exists

#   Failed test 'One failed notice currently exists'
#   at t/db_dependent/Koha/Notices.t line 98.
#          got: '0'
#     expected: '1'
ok 6 - No failed notices currently existing, now the notice has been marked pending
not ok 7 - Patron has a restriction placed on them

#   Failed test 'Patron has a restriction placed on them'
#   at t/db_dependent/Koha/Notices.t line 129.
#          got: '0'
#     expected: '1'
not ok 8 - No new restriction has been placed on the patron

#   Failed test 'No new restriction has been placed on the patron'
#   at t/db_dependent/Koha/Notices.t line 150.
#          got: '0'
#     expected: '1'
# Looks like you failed 3 tests of 8.
Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/8 subtests 

Test Summary Report
-------------------
t/db_dependent/Koha/Notices.t (Wstat: 768 Tests: 8 Failed: 3)
  Failed tests:  5, 7-8
  Non-zero exit status: 3
Files=1, Tests=8,  1 wallclock secs ( 0.02 usr  0.00 sys +  1.08 cusr  0.11 csys =  1.21 CPU)
Result: FAIL

ASSIGNED
Comment 5 Aleisha Amohia 2020-03-31 02:13:07 UTC
Created attachment 102098 [details] [review]
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail

When the 'RestrictPatronsWithFailedNotices' syspref is enabled then
patrons with email and sms notices which failed sending (have a
message_queue.status field of 'failed') have a restriction (debarment)
applied to them.

Test plan:
1. In the Koha staff client > Tools > Overdue notice/status triggers and
create the 'First' rule for all patron categories as:

Delay: 1
Letter: Overdue Notice
SMS: ticked

Ensure you have an SMS notice for the ODUE letter.

2. In the system preferences make sure you enter dummy data into the
SMSSendUsername, SMSSendPassword and EmailSMSSendDriverFromAddress
sysprefs

2. Find two non-debarred patrons and make sure they have invalid SMS numbers set. The SMS numbers must be INCORRECT, for example "123" as an SMS number. Leaving this field empty will result in the message_transport_type defaulting to 'print' instead of 'sms'.

3. Check one item out to each patron in step 2

4. Jump into the database and run the query:

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower1>;

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower2>;

5. Go to  misc/cronjobs directory and enter the Koha shell:

sudo koha-shell <instancename>

6. Run: ./overdue_notices.pl

7. Exit the shell and jump back into the database and run the query:

SELECT message_transport_type, status FROM message_queue WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

8. Confirm both new notice records have the message_transport_type is 'sms' and the status of 'pending'

9. Exit the database and re-enter the Koha shell and run the command:
./process_message_queue.pl

10. Jump back into the database re-run the query from step 7 and confirm
the status is 'failed' for both

11. Also run the query:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice there is no added debarment to these two patrons

12. Apply patch, restart memcached and plack. In the
installer/data/mysql directory enter the Koha shell and run the command:
./update_database.pl

13. In the Administration > Global System Preferences interface of the
staff client notice there is a new system (set to "Don't" by default)
named 'RestrictPatronsWithFailedNotices'.

Enable it (i.e. select 'Do')

14. Create a new file in the /etc/cron.daily directory named koha-custom and
add the following line to it:

koha-foreach --chdir --enabled
/usr/share/koha/bin/cronjobs/restrict_patrons_with_failed_notices.pl

15. In the  misc/cronjobs directory enter the Koha shell and run the command:
./restrict_patrons_with_failed_notices.pl

16. The script should output text saying: There are borrowers with
failed SMS or email notices

However because you haven't given the script the argument -c it won't
apply debarments (restrictions) to any of the patrons with the failed
SMS or email notices.

16. Query the borrower_debarments table:

SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice they still have no restriction

17. Now in the Koha shell run the command:
./restrict_patrons_with_failed_notices.pl -c

18. Notice the script outputs the text:
There are borrowers with failed SMS or email notices
Applying restriction to patron <borrowernumber>: <borrower firstname>
<borrower surname>;

19. Repeat step 16 and notice both patrons now have 1 restriction each
with the borrower_debarments.type=SUSPENSION and comment=SMSnumber
invalid and expiration=NULL

20. Query the borrowers table:
SELECT debarred, debarredcomment FROM borrowers WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

21. Notice the values are:
debarred= 9999-12-31
debarredcomment= SMS number invalid

22. Repeat step 17 and notice the script outputs:
There are borrowers with failed SMS or email notices

Patron <borrowernumber>: <borrower firstname> <borrower surname> is currently restricted due to having an invalid SMS number. No new restriction applied"

23. Repeat step 16 and notice no new debarment has been added to those
borrowers as they have already been restricted from having a failed SMS
notice.

24. In the Koha home directory run the command:
prove t/db_dependent/Koha/Notices.t

This unit test contains the tests for the new subroutines added to
Koha/Notice/Message.pm which are restrict_patron_when_notice_fails() and
get_failed_notices()

25. All tests should pass

26. Sign off

Sponsored-By: Brimbank Library, Australia
Comment 6 Bernardo Gonzalez Kriegel 2020-05-04 03:10:46 UTC
Hi,
patch does not apply, an easy conflict in patrons.pref

Solving that, qa tools little complain

 FAIL   misc/cronjobs/restrict_patrons_with_failed_notices.pl
   FAIL   critic
                "$type" is declared but not used at line 16, column 1. Unused variables clutter code and make it harder to read.
   OK     forbidden patterns
   OK     git manipulation
   OK     pod
   OK     spelling
   OK     valid

That new file also lacks a copyright notice!

Running that file always generates some output. It has a verbose option, but not a quiet/silent one. As a cron job it will always send an email with the output, unless you redirects out to /dev/null. 
Why not print something only with '-v' and be silent otherwise (unless there's some error)?

Not fully tested yet.
Comment 7 Aleisha Amohia 2020-05-18 06:03:25 UTC
Created attachment 105003 [details] [review]
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail

When the 'RestrictPatronsWithFailedNotices' syspref is enabled then
patrons with email and sms notices which failed sending (have a
message_queue.status field of 'failed') have a restriction (debarment)
applied to them.

Test plan:
1. In the Koha staff client > Tools > Overdue notice/status triggers and
create the 'First' rule for all patron categories as:

Delay: 1
Letter: Overdue Notice
SMS: ticked

Ensure you have an SMS notice for the ODUE letter.

2. In the system preferences make sure you enter dummy data into the
SMSSendUsername, SMSSendPassword and EmailSMSSendDriverFromAddress
sysprefs

2. Find two non-debarred patrons and make sure they have invalid SMS numbers set. The SMS numbers must be INCORRECT, for example "123" as an SMS number. Leaving this field empty will result in the message_transport_type defaulting to 'print' instead of 'sms'.

3. Check one item out to each patron in step 2

4. Jump into the database and run the query:

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower1>;

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower2>;

5. Go to  misc/cronjobs directory and enter the Koha shell:

sudo koha-shell <instancename>

6. Run: ./overdue_notices.pl

7. Exit the shell and jump back into the database and run the query:

SELECT message_transport_type, status FROM message_queue WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

8. Confirm both new notice records have the message_transport_type is 'sms' and the status of 'pending'

9. Exit the database and re-enter the Koha shell and run the command:
./process_message_queue.pl

10. Jump back into the database re-run the query from step 7 and confirm
the status is 'failed' for both

11. Also run the query:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice there is no added debarment to these two patrons

12. Apply patch, restart memcached and plack. In the
installer/data/mysql directory enter the Koha shell and run the command:
./update_database.pl

13. In the Administration > Global System Preferences interface of the
staff client notice there is a new system (set to "Don't" by default)
named 'RestrictPatronsWithFailedNotices'.

Enable it (i.e. select 'Do')

14. Create a new file in the /etc/cron.daily directory named koha-custom and
add the following line to it:

koha-foreach --chdir --enabled
/usr/share/koha/bin/cronjobs/restrict_patrons_with_failed_notices.pl

15. In the  misc/cronjobs directory enter the Koha shell and run the command:
./restrict_patrons_with_failed_notices.pl

16. The script should output text saying: There are borrowers with
failed SMS or email notices

However because you haven't given the script the argument -c it won't
apply debarments (restrictions) to any of the patrons with the failed
SMS or email notices.

16. Query the borrower_debarments table:

SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice they still have no restriction

17. Now in the Koha shell run the command:
./restrict_patrons_with_failed_notices.pl -c

18. Notice the script outputs the text:
There are borrowers with failed SMS or email notices
Applying restriction to patron <borrowernumber>: <borrower firstname>
<borrower surname>;

19. Repeat step 16 and notice both patrons now have 1 restriction each
with the borrower_debarments.type=SUSPENSION and comment=SMSnumber
invalid and expiration=NULL

20. Query the borrowers table:
SELECT debarred, debarredcomment FROM borrowers WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

21. Notice the values are:
debarred= 9999-12-31
debarredcomment= SMS number invalid

22. Repeat step 17 and notice the script outputs:
There are borrowers with failed SMS or email notices

Patron <borrowernumber>: <borrower firstname> <borrower surname> is currently restricted due to having an invalid SMS number. No new restriction applied"

23. Repeat step 16 and notice no new debarment has been added to those
borrowers as they have already been restricted from having a failed SMS
notice.

24. In the Koha home directory run the command:
prove t/db_dependent/Koha/Notices.t

This unit test contains the tests for the new subroutines added to
Koha/Notice/Message.pm which are restrict_patron_when_notice_fails() and
get_failed_notices()

25. All tests should pass

26. Sign off

Sponsored-By: Brimbank Library, Australia
Comment 8 Aleisha Amohia 2020-05-18 06:04:33 UTC
(In reply to Bernardo Gonzalez Kriegel from comment #6)
> Hi,
> patch does not apply, an easy conflict in patrons.pref
> 
> Solving that, qa tools little complain
> 
>  FAIL   misc/cronjobs/restrict_patrons_with_failed_notices.pl
>    FAIL   critic
>                 "$type" is declared but not used at line 16, column 1.
> Unused variables clutter code and make it harder to read.
>    OK     forbidden patterns
>    OK     git manipulation
>    OK     pod
>    OK     spelling
>    OK     valid
> 
> That new file also lacks a copyright notice!
> 
> Running that file always generates some output. It has a verbose option, but
> not a quiet/silent one. As a cron job it will always send an email with the
> output, unless you redirects out to /dev/null. 
> Why not print something only with '-v' and be silent otherwise (unless
> there's some error)?
> 
> Not fully tested yet.

Have fixed conflict, removed unused variable, added copyright notice, and amended the cron to default to no output unless the verbose option is specified. Ready for testing.
Comment 9 Fridolin Somers 2020-12-30 16:30:04 UTC
Nice work.

I just wonder about get_failed_notices() method in Koha/Notice/Message.pm.
It does not use current object.
And why using SQL and not the ORM ?

Also :
Koha::Notice::Message::restrict_patron_when_notice_fails($failed_notice);
This is strange.
Method restrict_patron_when_notice_fails() should be called on a Koha::Notice::Message object instance.
Comment 10 Aleisha Amohia 2021-02-01 03:19:54 UTC
I'll work on that ^
Comment 11 Aleisha Amohia 2021-02-01 20:56:07 UTC
Created attachment 116181 [details] [review]
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail

When the 'RestrictPatronsWithFailedNotices' syspref is enabled then
patrons with email and sms notices which failed sending (have a
message_queue.status field of 'failed') have a restriction (debarment)
applied to them.

Test plan:
1. In the Koha staff client > Tools > Overdue notice/status triggers and
create the 'First' rule for all patron categories as:

Delay: 1
Letter: Overdue Notice
SMS: ticked

Ensure you have an SMS notice for the ODUE letter.

2. In the system preferences make sure you enter dummy data into the
SMSSendUsername, SMSSendPassword and EmailSMSSendDriverFromAddress
sysprefs

2. Find two non-debarred patrons and make sure they have invalid SMS numbers set. The SMS numbers must be INCORRECT, for example "123" as an SMS number. Leaving this field empty will result in the message_transport_type defaulting to 'print' instead of 'sms'.

3. Check one item out to each patron in step 2

4. Jump into the database and run the query:

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower1>;

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower2>;

5. Go to  misc/cronjobs directory and enter the Koha shell:

sudo koha-shell <instancename>

6. Run: ./overdue_notices.pl

7. Exit the shell and jump back into the database and run the query:

SELECT message_transport_type, status FROM message_queue WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

8. Confirm both new notice records have the message_transport_type is 'sms' and the status of 'pending'

9. Exit the database and re-enter the Koha shell and run the command:
./process_message_queue.pl

10. Jump back into the database re-run the query from step 7 and confirm
the status is 'failed' for both

11. Also run the query:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice there is no added debarment to these two patrons

12. Apply patch, restart memcached and plack. In the
installer/data/mysql directory enter the Koha shell and run the command:
./update_database.pl

13. In the Administration > Global System Preferences interface of the
staff client notice there is a new system (set to "Don't" by default)
named 'RestrictPatronsWithFailedNotices'.

Enable it (i.e. select 'Do')

14. Create a new file in the /etc/cron.daily directory named koha-custom and
add the following line to it:

koha-foreach --chdir --enabled
/usr/share/koha/bin/cronjobs/restrict_patrons_with_failed_notices.pl

15. In the  misc/cronjobs directory enter the Koha shell and run the command:
./restrict_patrons_with_failed_notices.pl

16. The script should output text saying: There are borrowers with
failed SMS or email notices

However because you haven't given the script the argument -c it won't
apply debarments (restrictions) to any of the patrons with the failed
SMS or email notices.

16. Query the borrower_debarments table:

SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice they still have no restriction

17. Now in the Koha shell run the command:
./restrict_patrons_with_failed_notices.pl -c

18. Notice the script outputs the text:
There are borrowers with failed SMS or email notices
Applying restriction to patron <borrowernumber>: <borrower firstname>
<borrower surname>;

19. Repeat step 16 and notice both patrons now have 1 restriction each
with the borrower_debarments.type=SUSPENSION and comment=SMSnumber
invalid and expiration=NULL

20. Query the borrowers table:
SELECT debarred, debarredcomment FROM borrowers WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

21. Notice the values are:
debarred= 9999-12-31
debarredcomment= SMS number invalid

22. Repeat step 17 and notice the script outputs:
There are borrowers with failed SMS or email notices

Patron <borrowernumber>: <borrower firstname> <borrower surname> is currently restricted due to having an invalid SMS number. No new restriction applied"

23. Repeat step 16 and notice no new debarment has been added to those
borrowers as they have already been restricted from having a failed SMS
notice.

24. In the Koha home directory run the command:
prove t/db_dependent/Koha/Notices.t

This unit test contains the tests for the new subroutines added to
Koha/Notice/Message.pm which are restrict_patron_when_notice_fails() and
get_failed_notices()

25. All tests should pass

26. Sign off

Sponsored-By: Brimbank Library, Australia
Comment 12 Aleisha Amohia 2023-11-09 01:09:21 UTC
Created attachment 158687 [details] [review]
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail

When the 'RestrictPatronsWithFailedNotices' syspref is enabled then
patrons with email and sms notices which failed sending (have a
message_queue.status field of 'failed') have a restriction (debarment)
applied to them.

Test plan:
1. In the Koha staff client > Tools > Overdue notice/status triggers and
create the 'First' rule for all patron categories as:

Delay: 1
Letter: Overdue Notice
SMS: ticked

Ensure you have an SMS notice for the ODUE letter.

2. In the system preferences make sure you enter dummy data into the
SMSSendUsername, SMSSendPassword and EmailSMSSendDriverFromAddress
sysprefs

2. Find two non-debarred patrons and make sure they have invalid SMS numbers set. The SMS numbers must be INCORRECT, for example "123" as an SMS number. Leaving this field empty will result in the message_transport_type defaulting to 'print' instead of 'sms'.

3. Check one item out to each patron in step 2

4. Jump into the database and run the query:

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower1>;

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower2>;

5. Go to  misc/cronjobs directory and enter the Koha shell:

sudo koha-shell <instancename>

6. Run: ./overdue_notices.pl

7. Exit the shell and jump back into the database and run the query:

SELECT message_transport_type, status FROM message_queue WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

8. Confirm both new notice records have the message_transport_type is 'sms' and the status of 'pending'

9. Exit the database and re-enter the Koha shell and run the command:
./process_message_queue.pl

10. Jump back into the database re-run the query from step 7 and confirm
the status is 'failed' for both

11. Also run the query:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice there is no added debarment to these two patrons

12. Apply patch, restart memcached and plack. In the
installer/data/mysql directory enter the Koha shell and run the command:
./update_database.pl

13. In the Administration > Global System Preferences interface of the
staff client notice there is a new system (set to "Don't" by default)
named 'RestrictPatronsWithFailedNotices'.

Enable it (i.e. select 'Do')

14. Create a new file in the /etc/cron.daily directory named koha-custom and
add the following line to it:

koha-foreach --chdir --enabled
/usr/share/koha/bin/cronjobs/restrict_patrons_with_failed_notices.pl

15. In the  misc/cronjobs directory enter the Koha shell and run the command:
./restrict_patrons_with_failed_notices.pl

16. The script should output text saying: There are borrowers with
failed SMS or email notices

However because you haven't given the script the argument -c it won't
apply debarments (restrictions) to any of the patrons with the failed
SMS or email notices.

16. Query the borrower_debarments table:

SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice they still have no restriction

17. Now in the Koha shell run the command:
./restrict_patrons_with_failed_notices.pl -c

18. Notice the script outputs the text:
There are borrowers with failed SMS or email notices
Applying restriction to patron <borrowernumber>: <borrower firstname>
<borrower surname>;

19. Repeat step 16 and notice both patrons now have 1 restriction each
with the borrower_debarments.type=SUSPENSION and comment=SMSnumber
invalid and expiration=NULL

20. Query the borrowers table:
SELECT debarred, debarredcomment FROM borrowers WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

21. Notice the values are:
debarred= 9999-12-31
debarredcomment= SMS number invalid

22. Repeat step 17 and notice the script outputs:
There are borrowers with failed SMS or email notices

Patron <borrowernumber>: <borrower firstname> <borrower surname> is currently restricted due to having an invalid SMS number. No new restriction applied"

23. Repeat step 16 and notice no new debarment has been added to those
borrowers as they have already been restricted from having a failed SMS
notice.

24. In the Koha home directory run the command:
prove t/db_dependent/Koha/Notices.t

This unit test contains the tests for the new subroutines added to
Koha/Notice/Message.pm which are restrict_patron_when_notice_fails() and
get_failed_notices()

25. All tests should pass

26. Sign off

Sponsored-by: Catalyst IT
Comment 13 Matthias Le Gac 2024-01-09 19:48:51 UTC
When I applied the patch it doesnt't work. I have this error :

Applying: Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail
Using index info to reconstruct a base tree...
M	Koha/Notice/Message.pm
M	installer/data/mysql/mandatory/sysprefs.sql
M	koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/patrons.pref
Falling back to patching base and 3-way merge...
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/patrons.pref
CONFLICT (content): Merge conflict in koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/patrons.pref
Auto-merging installer/data/mysql/mandatory/sysprefs.sql
Auto-merging Koha/Notice/Message.pm
CONFLICT (content): Merge conflict in Koha/Notice/Message.pm
error: Failed to merge in the changes.
Patch failed at 0001 Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail
Comment 14 Aleisha Amohia 2024-01-10 21:18:01 UTC
Created attachment 160785 [details] [review]
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail

When the 'RestrictPatronsWithFailedNotices' syspref is enabled then
patrons with email and sms notices which failed sending (have a
message_queue.status field of 'failed') have a restriction (debarment)
applied to them.

Test plan:
1. In the Koha staff client > Tools > Overdue notice/status triggers and
create the 'First' rule for all patron categories as:

Delay: 1
Letter: Overdue Notice
SMS: ticked

Ensure you have an SMS notice for the ODUE letter.

2. In the system preferences make sure you enter dummy data into the
SMSSendUsername, SMSSendPassword and EmailSMSSendDriverFromAddress
sysprefs

2. Find two non-debarred patrons and make sure they have invalid SMS numbers set. The SMS numbers must be INCORRECT, for example "123" as an SMS number. Leaving this field empty will result in the message_transport_type defaulting to 'print' instead of 'sms'.

3. Check one item out to each patron in step 2

4. Jump into the database and run the query:

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower1>;

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower2>;

5. Go to  misc/cronjobs directory and enter the Koha shell:

sudo koha-shell <instancename>

6. Run: ./overdue_notices.pl

7. Exit the shell and jump back into the database and run the query:

SELECT message_transport_type, status FROM message_queue WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

8. Confirm both new notice records have the message_transport_type is 'sms' and the status of 'pending'

9. Exit the database and re-enter the Koha shell and run the command:
./process_message_queue.pl

10. Jump back into the database re-run the query from step 7 and confirm
the status is 'failed' for both

11. Also run the query:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice there is no added debarment to these two patrons

12. Apply patch, restart memcached and plack. In the
installer/data/mysql directory enter the Koha shell and run the command:
./update_database.pl

13. In the Administration > Global System Preferences interface of the
staff client notice there is a new system (set to "Don't" by default)
named 'RestrictPatronsWithFailedNotices'.

Enable it (i.e. select 'Do')

14. Create a new file in the /etc/cron.daily directory named koha-custom and
add the following line to it:

koha-foreach --chdir --enabled
/usr/share/koha/bin/cronjobs/restrict_patrons_with_failed_notices.pl

15. In the  misc/cronjobs directory enter the Koha shell and run the command:
./restrict_patrons_with_failed_notices.pl

16. The script should output text saying: There are borrowers with
failed SMS or email notices

However because you haven't given the script the argument -c it won't
apply debarments (restrictions) to any of the patrons with the failed
SMS or email notices.

16. Query the borrower_debarments table:

SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice they still have no restriction

17. Now in the Koha shell run the command:
./restrict_patrons_with_failed_notices.pl -c

18. Notice the script outputs the text:
There are borrowers with failed SMS or email notices
Applying restriction to patron <borrowernumber>: <borrower firstname>
<borrower surname>;

19. Repeat step 16 and notice both patrons now have 1 restriction each
with the borrower_debarments.type=SUSPENSION and comment=SMSnumber
invalid and expiration=NULL

20. Query the borrowers table:
SELECT debarred, debarredcomment FROM borrowers WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

21. Notice the values are:
debarred= 9999-12-31
debarredcomment= SMS number invalid

22. Repeat step 17 and notice the script outputs:
There are borrowers with failed SMS or email notices

Patron <borrowernumber>: <borrower firstname> <borrower surname> is currently restricted due to having an invalid SMS number. No new restriction applied"

23. Repeat step 16 and notice no new debarment has been added to those
borrowers as they have already been restricted from having a failed SMS
notice.

24. In the Koha home directory run the command:
prove t/db_dependent/Koha/Notices.t

This unit test contains the tests for the new subroutines added to
Koha/Notice/Message.pm which are restrict_patron_when_notice_fails() and
get_failed_notices()

25. All tests should pass

26. Sign off

Sponsored-by: Catalyst IT
Comment 15 Lucas Gass 2024-01-10 23:33:02 UTC
Aleisha,

There is a YAML problem with patrons.pref, I believe. With this patchset applied when I do a seach for system preferences I get:

YAML::XS::Load Error: The problem:

    did not find expected key

was found at document: 1, line: 391, column: 5
while parsing a block mapping at line: 2, column: 5
 at /kohadevbox/koha/admin/preferences.pl line 51

    in CGI::Compile::ROOT::kohadevbox_koha_admin_preferences_2epl::GetTab at /kohadevbox/koha/admin/preferences.pl line 51 


Also, ,maybe perltidy "restrict_patrons_with_failed_notices.pl" ?
Comment 16 Aleisha Amohia 2024-01-11 19:07:58 UTC
Created attachment 160890 [details] [review]
Bug 23295: Automatically restrict (debar) patrons when email/sms notices fail

When the 'RestrictPatronsWithFailedNotices' syspref is enabled then
patrons with email and sms notices which failed sending (have a
message_queue.status field of 'failed') have a restriction (debarment)
applied to them.

Test plan:
1. In the Koha staff client > Tools > Overdue notice/status triggers and
create the 'First' rule for all patron categories as:

Delay: 1
Letter: Overdue Notice
SMS: ticked

Ensure you have an SMS notice for the ODUE letter.

2. In the system preferences make sure you enter dummy data into the
SMSSendUsername, SMSSendPassword and EmailSMSSendDriverFromAddress
sysprefs

2. Find two non-debarred patrons and make sure they have invalid SMS numbers set. The SMS numbers must be INCORRECT, for example "123" as an SMS number. Leaving this field empty will result in the message_transport_type defaulting to 'print' instead of 'sms'.

3. Check one item out to each patron in step 2

4. Jump into the database and run the query:

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower1>;

UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower2>;

5. Go to  misc/cronjobs directory and enter the Koha shell:

sudo koha-shell <instancename>

6. Run: ./overdue_notices.pl

7. Exit the shell and jump back into the database and run the query:

SELECT message_transport_type, status FROM message_queue WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

8. Confirm both new notice records have the message_transport_type is 'sms' and the status of 'pending'

9. Exit the database and re-enter the Koha shell and run the command:
./process_message_queue.pl

10. Jump back into the database re-run the query from step 7 and confirm
the status is 'failed' for both

11. Also run the query:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice there is no added debarment to these two patrons

12. Apply patch, restart memcached and plack. In the
installer/data/mysql directory enter the Koha shell and run the command:
./update_database.pl

13. In the Administration > Global System Preferences interface of the
staff client notice there is a new system (set to "Don't" by default)
named 'RestrictPatronsWithFailedNotices'.

Enable it (i.e. select 'Do')

14. Create a new file in the /etc/cron.daily directory named koha-custom and
add the following line to it:

koha-foreach --chdir --enabled
/usr/share/koha/bin/cronjobs/restrict_patrons_with_failed_notices.pl

15. In the  misc/cronjobs directory enter the Koha shell and run the command:
./restrict_patrons_with_failed_notices.pl

16. The script should output text saying: There are borrowers with
failed SMS or email notices

However because you haven't given the script the argument -c it won't
apply debarments (restrictions) to any of the patrons with the failed
SMS or email notices.

16. Query the borrower_debarments table:

SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;

Notice they still have no restriction

17. Now in the Koha shell run the command:
./restrict_patrons_with_failed_notices.pl -c

18. Notice the script outputs the text:
There are borrowers with failed SMS or email notices
Applying restriction to patron <borrowernumber>: <borrower firstname>
<borrower surname>;

19. Repeat step 16 and notice both patrons now have 1 restriction each
with the borrower_debarments.type=SUSPENSION and comment=SMSnumber
invalid and expiration=NULL

20. Query the borrowers table:
SELECT debarred, debarredcomment FROM borrowers WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;

21. Notice the values are:
debarred= 9999-12-31
debarredcomment= SMS number invalid

22. Repeat step 17 and notice the script outputs:
There are borrowers with failed SMS or email notices

Patron <borrowernumber>: <borrower firstname> <borrower surname> is currently restricted due to having an invalid SMS number. No new restriction applied"

23. Repeat step 16 and notice no new debarment has been added to those
borrowers as they have already been restricted from having a failed SMS
notice.

24. In the Koha home directory run the command:
prove t/db_dependent/Koha/Notices.t

This unit test contains the tests for the new subroutines added to
Koha/Notice/Message.pm which are restrict_patron_when_notice_fails() and
get_failed_notices()

25. All tests should pass

26. Sign off

Sponsored-by: Catalyst IT