Comment on attachment 53354 [details] [review]
Bug 15986 - Add a script for sending hold waiting reminder notices

Review of attachment 53354 [details] [review]:
-----------------------------------------------------------------

::: misc/cronjobs/holds_reminder.pl
@@ +1,4 @@
> +#!/usr/bin/perl
> +
> +# Copyright 2008 Liblime
> +# Copyright 2010 BibLibre

Can we please fix the copyright notices, this is a new script right?

Just a general comment, we have a setting where borrowers can opt in/out of holds filled notices.
Should this script respect that preference, IE if they don't get a hold filled notice will they be happy getting one saying it's waiting. Annoying patrons is something libraries generally like to avoid :)

It should be easy enough to use the check we have for holds filled in circulation I hope

Created attachment 53594 [details] [review]
Bug 15986 - Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.
Essentially it is a modification of the overdue_notices.pl script

To test:
1 - Create a hold reminder notice (Koha module: Holds)
2 - Set some waiting holds for patrons
3 - Run the script with various options and verify correct data is
    generated
4 - Verify that with 'optout' switch set only patrons who have email
hold filled notices specified will recieve a reminder

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

(In reply to Chris Cormack from comment #4)
> Just a general comment, we have a setting where borrowers can opt in/out of
> holds filled notices.
> Should this script respect that preference, IE if they don't get a hold
> filled notice will they be happy getting one saying it's waiting. Annoying
> patrons is something libraries generally like to avoid :)

Good point, I added it as a switch, some libraries charge fines for holds not picked up etc, so they may want to force the notification, I figure this way makes it flexible

Yep sounds good, I'll give it a test when I'm back at work

Created attachment 55723 [details] [review]
[SIGNED-OFF] Bug 15986 - Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.
Essentially it is a modification of the overdue_notices.pl script

To test:
1 - Create a hold reminder notice (Koha module: Holds)
2 - Set some waiting holds for patrons
3 - Run the script with various options and verify correct data is
    generated
4 - Verify that with 'optout' switch set only patrons who have email
hold filled notices specified will recieve a reminder

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

Signed-off-by: Robert <rhilliker@hotchkiss.org>

I love the amount of documentation for this script!

I didn't get to test this yet, but while reading through it I wondered:
There is a lot of work done in order to allow exporting in different formats. I was wondering if it might be better to leave the job of creating a printable version to gather_print_notices.pl working towards a more unified workflow for all the printed letters.

I like the feature, but would love to see less (new) code in cron scripts and more code in our modules. With advantages of testing etc.

This is a really exciting feature.

I know it's been sent back to discussion from being signed off - what's the next step for moving it forward?  We think this would make a fantastic addition to Koha.

(In reply to Sally Healey from comment #11)
> This is a really exciting feature.
> 
> I know it's been sent back to discussion from being signed off - what's the
> next step for moving it forward?  We think this would make a fantastic
> addition to Koha.

To move it forward it needs comment 10 addressed, and a lot of the code moved out of the .pl file into a module, with unit tests written. 
This shouldn't be a big job, but will mean way less chance of breakage/maintenance in the future.
As well as a signoff from outside of the sponsoring library.

I think comment 9 could be a future enhancement. But mostly comment 10 (moving the code to a module and writing tests) and an independent sign off are what are need

I think if we do #9 later it will introduce a change in workflow - if we did it from the beginning, we could safe people from having to change their setups. But I am not blocking this, just explaining my point of view maybe also for future enh.

Created attachment 91767 [details] [review]
Bug 15986: Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.

We add a 'send_notice' routine to Koha::Patrons - this will either send using the patron's
email prefs, or allow forcing of a single method via the cron

To test:
 1 - Create an email hold reminder notice for a single library (Koha module: Holds, code HOLDREMINDER, branch: CPL)
 2 - Set some waiting holds today for patrons at CPL, ensure those patrons have 'email' as the transport for hold filled notices
 3 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL
 4 - You should see the patrons here would have received emails
 5 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL
 6 - You should see the emails that were sent
 7 - Check the patron notices tab to confirm
 8 - Note a ptron with two holds waiting receives only one notice
 9 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
10 - No notices are sent
11 - Adjust the waiting date for the holds:
    UPDATE reserves SET waitingdate=DATE_SUB(CURDATE(), INTERVAL 3 DAYS) WHERE waitingdate = CURDATE();
12 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
13 - Confirm the holds are now reminded
14 - Set yesterday as a holiday for CPL
15 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 3
16 - Notices should not be sent
17 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 4
18 - Notices should be sent again
19 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4
20 - Should get feedback that notice was not found for other libraries
21 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
22 - Notice is not found
23 - Add the notice for sms
24 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
25 - The notice should be sent
26 - Check patrons messaging tab to confirm
27 - prove -v t/db_dependent/Koha/Patrons.t

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

(In reply to Chris Cormack from comment #12)
> I think comment 9 could be a future enhancement. But mostly comment 10
> (moving the code to a module and writing tests) and an independent sign off
> are what are need

Code for sending moved to Koha::Patron::send_notice

(In reply to Katrin Fischer from comment #13)
> I think if we do #9 later it will introduce a change in workflow - if we did
> it from the beginning, we could safe people from having to change their
> setups. But I am not blocking this, just explaining my point of view maybe
> also for future enh.

Removed the option to print out notices, now fallback to print if no email/sms

Created attachment 91768 [details] [review]
Bug 15986: Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.

We add a 'send_notice' routine to Koha::Patrons - this will either send using the patron's
email prefs, or allow forcing of a single method via the cron

To test:
 1 - Create an email hold reminder notice for a single library (Koha module: Holds, code HOLDREMINDER, branch: CPL)
 2 - Set some waiting holds today for patrons at CPL, ensure those patrons have 'email' as the transport for hold filled notices
 3 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL
 4 - You should see the patrons here would have received emails
 5 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL
 6 - You should see the emails that were sent
 7 - Check the patron notices tab to confirm
 8 - Note a ptron with two holds waiting receives only one notice
 9 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
10 - No notices are sent
11 - Adjust the waiting date for the holds:
    UPDATE reserves SET waitingdate=DATE_SUB(CURDATE(), INTERVAL 3 DAYS) WHERE waitingdate = CURDATE();
12 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
13 - Confirm the holds are now reminded
14 - Set yesterday as a holiday for CPL
15 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 3
16 - Notices should not be sent
17 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 4
18 - Notices should be sent again
19 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4
20 - Should get feedback that notice was not found for other libraries
21 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
22 - Notice is not found
23 - Add the notice for sms
24 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
25 - The notice should be sent
26 - Check patrons messaging tab to confirm
27 - prove -v t/db_dependent/Koha/Patrons.t

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

Patch no longer applies

Created attachment 99919 [details] [review]
Bug 15986: Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.

We add a 'send_notice' routine to Koha::Patrons - this will either send using the patron's
email prefs, or allow forcing of a single method via the cron

To test:
 1 - Create an email hold reminder notice for a single library (Koha module: Holds, code HOLDREMINDER, branch: CPL)
 2 - Set some waiting holds today for patrons at CPL, ensure those patrons have 'email' as the transport for hold filled notices
 3 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL
 4 - You should see the patrons here would have received emails
 5 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL
 6 - You should see the emails that were sent
 7 - Check the patron notices tab to confirm
 8 - Note a ptron with two holds waiting receives only one notice
 9 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
10 - No notices are sent
11 - Adjust the waiting date for the holds:
    UPDATE reserves SET waitingdate=DATE_SUB(CURDATE(), INTERVAL 3 DAYS) WHERE waitingdate = CURDATE();
12 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
13 - Confirm the holds are now reminded
14 - Set yesterday as a holiday for CPL
15 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 3
16 - Notices should not be sent
17 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 4
18 - Notices should be sent again
19 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4
20 - Should get feedback that notice was not found for other libraries
21 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
22 - Notice is not found
23 - Add the notice for sms
24 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
25 - The notice should be sent
26 - Check patrons messaging tab to confirm
27 - prove -v t/db_dependent/Koha/Patrons.t

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

Is the test_mode parameter of the method really useful?

(In reply to Jonathan Druart from comment #20)
> Is the test_mode parameter of the method really useful?

Yes.

Things move too fast 8-) - patch no longer applies:

root@5e1d1195a414:koha(bz15986)$ git bz apply 15986
Bug 15986 - Add a script for sending hold waiting reminder notices

99919 - Bug 15986: Add a script for sending hold waiting reminder notices

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 15986: Add a script for sending hold waiting reminder notices
Using index info to reconstruct a base tree...
M	C4/Letters.pm
M	Koha/Patron.pm
M	t/db_dependent/Koha/Patrons.t
Falling back to patching base and 3-way merge...
Auto-merging t/db_dependent/Koha/Patrons.t
CONFLICT (content): Merge conflict in t/db_dependent/Koha/Patrons.t
Auto-merging Koha/Patron.pm
CONFLICT (content): Merge conflict in Koha/Patron.pm
Auto-merging C4/Letters.pm
error: Failed to merge in the changes.
Patch failed at 0001 Bug 15986: Add a script for sending hold waiting reminder notices

Also noticed that there is a holds subfolder in misc/cronjobs - should the script go in that?

Created attachment 102723 [details] [review]
Bug 15986: Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.

We add a 'send_notice' routine to Koha::Patrons - this will either send using the patron's
email prefs, or allow forcing of a single method via the cron

To test:
 1 - Create an email hold reminder notice for a single library (Koha module: Holds, code HOLDREMINDER, branch: CPL)
 2 - Set some waiting holds today for patrons at CPL, ensure those patrons have 'email' as the transport for hold filled notices
 3 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL
 4 - You should see the patrons here would have received emails
 5 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL
 6 - You should see the emails that were sent
 7 - Check the patron notices tab to confirm
 8 - Note a ptron with two holds waiting receives only one notice
 9 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
10 - No notices are sent
11 - Adjust the waiting date for the holds:
    UPDATE reserves SET waitingdate=DATE_SUB(CURDATE(), INTERVAL 3 DAYS) WHERE waitingdate = CURDATE();
12 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
13 - Confirm the holds are now reminded
14 - Set yesterday as a holiday for CPL
15 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 3
16 - Notices should not be sent
17 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 4
18 - Notices should be sent again
19 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4
20 - Should get feedback that notice was not found for other libraries
21 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
22 - Notice is not found
23 - Add the notice for sms
24 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
25 - The notice should be sent
26 - Check patrons messaging tab to confirm
27 - prove -v t/db_dependent/Koha/Patrons.t

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

For step 17, 19, 21, and 24 I'm getting an error message when running the script:

perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 3
branch CPL passed on parameter
Can't locate object method "in_units" via package "-3" (perhaps you forgot to load "-3"?) at /kohadevbox/koha/Koha/Calendar.pm line 193.

Also, in step 11 '..INTERVAL 3 DAYS)..' should be '..INTERVAL 3 DAY)..' (that is, DAYS should be DAY).

Created attachment 102926 [details] [review]
Bug 15986: Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.

We add a 'send_notice' routine to Koha::Patrons - this will either send using the patron's
email prefs, or allow forcing of a single method via the cron

To test:
 1 - Create an email hold reminder notice for a single library (Koha module: Holds, code HOLDREMINDER, branch: CPL)
 2 - Set some waiting holds today for patrons at CPL, ensure those patrons have 'email' as the transport for hold filled notices
 3 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL
 4 - You should see the patrons here would have received emails
 5 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL
 6 - You should see the emails that were sent
 7 - Check the patron notices tab to confirm
 8 - Note a ptron with two holds waiting receives only one notice
 9 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
10 - No notices are sent
11 - Adjust the waiting date for the holds:
    UPDATE reserves SET waitingdate=DATE_SUB(CURDATE(), INTERVAL 3 DAY) WHERE waitingdate = CURDATE();
12 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
13 - Confirm the holds are now reminded
14 - Set yesterday as a holiday for CPL
15 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 3
16 - Notices should not be sent
17 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 4
18 - Notices should be sent again
19 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4
20 - Should get feedback that notice was not found for other libraries
21 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
22 - Notice is not found
23 - Add the notice for sms
24 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
25 - The notice should be sent
26 - Check patrons messaging tab to confirm
27 - prove -v t/db_dependent/Koha/Patrons.t

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

(In reply to David Nind from comment #24)
> For step 17, 19, 21, and 24 I'm getting an error message when running the
> script:
> 
> perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL
> -holidays -days 3
> branch CPL passed on parameter
> Can't locate object method "in_units" via package "-3" (perhaps you forgot
> to load "-3"?) at /kohadevbox/koha/Koha/Calendar.pm line 193.

Fixed, thanks David!

Thanks Nick!

I can now get most things to work, except for steps 17 and 21 - not sure what I'm doing wrong here.

There are reserves table entries: current date = 2020-04-15, waitingdate for items = 2020-04-12, holiday date set (day = 14, month =4, yeare = 20204

Step 17:

perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 4
branch CPL passed on parameter
No reserves found for CPL

Step 21:

perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 4 -mtt sms
No reserves found for CPL
Message 'HOLDREMINDER' content not found for FFL
Message 'HOLDREMINDER' content not found for FPL
...

Comment on attachment 102926 [details] [review]
Bug 15986: Add a script for sending hold waiting reminder notices

Review of attachment 102926 [details] [review]:
-----------------------------------------------------------------

::: Koha/Patron.pm
@@ +1695,4 @@
>      };
>  }
>  
> +=head3 send_notice

Hmm, feels like this should be 'queue_notice' rather than 'send_notice'.. that paves the way for a later 'send_notice' which sends immediately vs waiting in the queue.

(In reply to Martin Renvoize from comment #29)
> Comment on attachment 102926 [details] [review] [review]
> Bug 15986: Add a script for sending hold waiting reminder notices
> 
> Review of attachment 102926 [details] [review] [review]:
> -----------------------------------------------------------------
> 
> ::: Koha/Patron.pm
> @@ +1695,4 @@
> >      };
> >  }
> >  
> > +=head3 send_notice
> 
> Hmm, feels like this should be 'queue_notice' rather than 'send_notice'..
> that paves the way for a later 'send_notice' which sends immediately vs
> waiting in the queue.

We already have a way to send directly without waiting through the queue - it's used by the self registration and password reset emails: bug 21390

+            my $num_letters_attempted = C4::Letters::SendQueuedMessages( {
+                    letter_code => 'OPAC_REG_VERIFY'
+                    } );

I'm also getting stuck at step 17. I have my reserves with waiting date set to curdate-3. They properly trigger a notice when not worrying about holidays. But once I set the -holidays flag in the command I cannot get notices to generate, no matter how many days I give it.

Created attachment 106962 [details] [review]
Bug 15986: Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.

We add a 'send_notice' routine to Koha::Patrons - this will either send using the patron's
email prefs, or allow forcing of a single method via the cron

To test:
 1 - Create an email hold reminder notice for a single library (Koha module: Holds, code HOLDREMINDER, branch: CPL)
 2 - Set some waiting holds today for patrons at CPL, ensure those patrons have 'email' as the transport for hold filled notices
 3 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL
 4 - You should see the patrons here would have received emails
 5 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL
 6 - You should see the emails that were sent
 7 - Check the patron notices tab to confirm
 8 - Note a ptron with two holds waiting receives only one notice
 9 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
10 - No notices are sent
11 - Adjust the waiting date for the holds:
    UPDATE reserves SET waitingdate=DATE_SUB(CURDATE(), INTERVAL 3 DAY) WHERE waitingdate = CURDATE();
12 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
13 - Confirm the holds are now reminded
14 - Set yesterday as a holiday for CPL
15 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 3
16 - Notices should not be sent
17 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 2
18 - Notices should be sent again
19 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 2
20 - Should get feedback that notice was not found for other libraries
21 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 2 -mtt sms
22 - Notice is not found
23 - Add the notice for sms
24 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 2 -mtt sms
25 - The notice should be sent
26 - Check patrons messaging tab to confirm
27 - prove -v t/db_dependent/Koha/Patrons.t

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

Created attachment 106963 [details] [review]
Bug 15986: Update for bug 24159

The job is concerned with holds waiting and takes days, I think using days_mode 'Calendar'
makes sense as we are not calculating due dates

Created attachment 106964 [details] [review]
Bug 15986: Rename send_notice to queue_notice and update test count

Adding a holiday means the holds are waiting 1-less day, not 1 more :-)

Fixed the math in the test plan :-)

This patch works fine !
But I won't sign it because of 2 points : 
- I can't test the sms part, I don't know how it works and how to use the feature
- I've got a question about the content of the message : when there several holds, only the informations about the oldest reserve (for what I saw) are displayed. Fields coming from the items or biblio tables can't be used. I know it's happening for some other notices so don't Failed the patch because of this point, but I regret I couldn't use the <<count>> or <<items.content>> datas as it is recommended for some other notices, I think it could be useful for patron to have some details about the waiting holds.

Created attachment 112999 [details] [review]
Bug 15986: Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.

We add a 'send_notice' routine to Koha::Patrons - this will either send using the patron's
email prefs, or allow forcing of a single method via the cron

To test:
 1 - Create an email hold reminder notice for a single library (Koha module: Holds, code HOLDREMINDER, branch: CPL)
 2 - Set some waiting holds today for patrons at CPL, ensure those patrons have 'email' as the transport for hold filled notices
 3 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL
 4 - You should see the patrons here would have received emails
 5 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL
 6 - You should see the emails that were sent
 7 - Check the patron notices tab to confirm
 8 - Note a ptron with two holds waiting receives only one notice
 9 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
10 - No notices are sent
11 - Adjust the waiting date for the holds:
    UPDATE reserves SET waitingdate=DATE_SUB(CURDATE(), INTERVAL 3 DAY) WHERE waitingdate = CURDATE();
12 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
13 - Confirm the holds are now reminded
14 - Set yesterday as a holiday for CPL
15 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 3
16 - Notices should not be sent
17 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 2
18 - Notices should be sent again
19 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 2
20 - Should get feedback that notice was not found for other libraries
21 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 2 -mtt sms
22 - Notice is not found
23 - Add the notice for sms
24 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 2 -mtt sms
25 - The notice should be sent
26 - Check patrons messaging tab to confirm
27 - prove -v t/db_dependent/Koha/Patrons.t

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

Signed-off-by: Kim Gnerre <kgnerre@hotchkiss.org>

Created attachment 113000 [details] [review]
Bug 15986: Update for bug 24159

The job is concerned with holds waiting and takes days, I think using days_mode 'Calendar'
makes sense as we are not calculating due dates

Signed-off-by: Kim Gnerre <kgnerre@hotchkiss.org>

Created attachment 113001 [details] [review]
Bug 15986: Rename send_notice to queue_notice and update test count

Signed-off-by: Kim Gnerre <kgnerre@hotchkiss.org>

(In reply to Séverine Queune from comment #36)
> This patch works fine !
> But I won't sign it because of 2 points : 
> - I can't test the sms part, I don't know how it works and how to use the
> feature
> - I've got a question about the content of the message : when there several
> holds, only the informations about the oldest reserve (for what I saw) are
> displayed. Fields coming from the items or biblio tables can't be used. I
> know it's happening for some other notices so don't Failed the patch because
> of this point, but I regret I couldn't use the <<count>> or
> <<items.content>> datas as it is recommended for some other notices, I think
> it could be useful for patron to have some details about the waiting holds.

- sms part works exactly like email up to the sending of the message, so testing is the same, just need to confirm it makes it into the queue

- the notice supports template toolkit, so you can get other holds information like:
Dear [% borrower.firstname %] [% borrower.surname %],

Your hold for [% hold.biblio.title %] is waiting since [% hold.waitingdate %]

[% IF borrower.holds.count > 0 %]
  You also have other holds:
  [% FOREACH hold IN borrower.holds %]
    [% hold.biblio.title %] : [% hold.found %]
  [% END %]
[% END %]

That could be cleaned up and made fancier, but just to give an idea

(In reply to Nick Clemens from comment #40)
> (In reply to Séverine Queune from comment #36)
> > This patch works fine !
> > But I won't sign it because of 2 points : 
> > - I can't test the sms part, I don't know how it works and how to use the
> > feature
> > - I've got a question about the content of the message : when there several
> > holds, only the informations about the oldest reserve (for what I saw) are
> > displayed. Fields coming from the items or biblio tables can't be used. I
> > know it's happening for some other notices so don't Failed the patch because
> > of this point, but I regret I couldn't use the <<count>> or
> > <<items.content>> datas as it is recommended for some other notices, I think
> > it could be useful for patron to have some details about the waiting holds.
> 
> - sms part works exactly like email up to the sending of the message, so
> testing is the same, just need to confirm it makes it into the queue
> 
> - the notice supports template toolkit, so you can get other holds
> information like:
> Dear [% borrower.firstname %] [% borrower.surname %],
> 
> Your hold for [% hold.biblio.title %] is waiting since [% hold.waitingdate %]
> 
> [% IF borrower.holds.count > 0 %]
>   You also have other holds:
>   [% FOREACH hold IN borrower.holds %]
>     [% hold.biblio.title %] : [% hold.found %]
>   [% END %]
> [% END %]
> 
> That could be cleaned up and made fancier, but just to give an idea

Thanks Nick !
I completely forgot we could use template toolkit
#ashamed... 
Do you need a second sign off for that one ?

Can't QA this one but got a question about -holidays, why is it better to not take them into account by default? I don't have much functional experience so I might miss something obvious ^^

The various sysprefs that determine whether or not to consult the calendar for various tasks are inconsistent in a new install -- some are on and some are off. However, ExcludeHolidaysFromMaxPickUpDelay defaults to off. That's not exactly the same as the --holiday flag here, but seems close enough that I'm happy to make this new feature keep the same default choice.

> The various sysprefs that determine whether or not to consult the calendar for various tasks are inconsistent in a new install

:o

> However, ExcludeHolidaysFromMaxPickUpDelay defaults to off. That's not exactly the same as the --holiday flag here, but seems close enough that I'm happy to make this new feature keep the same default choice.

Thanks for the analysis. Is there an obviously less surprising default choice there? Or does it depend too much?

I don't really get the sense one option is more standard or more likely to be correct for any given library -- it's just one of those things one ends up asking every new library about. Same with UseDaysMode, finesCalendar, OverdueNoticeCalendar, etc.

Good, there doesn't seem to be a bad choice.
Thanks for the info Andrew, that was my 2 cents, I'll leave the rest to more knowledgeable people ^^"

Created attachment 118733 [details] [review]
Bug 15986: Add a script for sending hold waiting reminder notices

This patch adds a script for sending holds reminder notice to patrons.

We add a 'send_notice' routine to Koha::Patrons - this will either send using the patron's
email prefs, or allow forcing of a single method via the cron

To test:
 1 - Create an email hold reminder notice for a single library (Koha module: Holds, code HOLDREMINDER, branch: CPL)
 2 - Set some waiting holds today for patrons at CPL, ensure those patrons have 'email' as the transport for hold filled notices
 3 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL
 4 - You should see the patrons here would have received emails
 5 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL
 6 - You should see the emails that were sent
 7 - Check the patron notices tab to confirm
 8 - Note a ptron with two holds waiting receives only one notice
 9 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
10 - No notices are sent
11 - Adjust the waiting date for the holds:
    UPDATE reserves SET waitingdate=DATE_SUB(CURDATE(), INTERVAL 3 DAY) WHERE waitingdate = CURDATE();
12 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -li CPL -days 3
13 - Confirm the holds are now reminded
14 - Set yesterday as a holiday for CPL
15 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 3
16 - Notices should not be sent
17 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -li CPL -holidays -days 2
18 - Notices should be sent again
19 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 2
20 - Should get feedback that notice was not found for other libraries
21 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 2 -mtt sms
22 - Notice is not found
23 - Add the notice for sms
24 - perl misc/cronjobs/holds_reminder.pl -v -lettercode HOLDREMINDER -n -holidays -days 2 -mtt sms
25 - The notice should be sent
26 - Check patrons messaging tab to confirm
27 - prove -v t/db_dependent/Koha/Patrons.t

Sponsored by: The Hotchkiss School (http://www.hotchkiss.org/)

Signed-off-by: Kim Gnerre <kgnerre@hotchkiss.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Created attachment 118734 [details] [review]
Bug 15986: Update for bug 24159

The job is concerned with holds waiting and takes days, I think using days_mode 'Calendar'
makes sense as we are not calculating due dates

Signed-off-by: Kim Gnerre <kgnerre@hotchkiss.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Created attachment 118735 [details] [review]
Bug 15986: Rename send_notice to queue_notice and update test count

Signed-off-by: Kim Gnerre <kgnerre@hotchkiss.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Created attachment 118736 [details] [review]
Bug 15986: (QA follow-up) Account for bugs 25333 and 25334

Bug 25333 changed the existing 'phone' message transport type to 'itiva' and
bug 25334 added a new 'phone' message transport type.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Whilst I feel we could continue to iterate on this and make improvements, I think this enhancement has waited long enough and those iterations can come in the future as the queue_notice method starts to get more widely adopted.

It all works as expected, contained tests for the new Patron method and causes no regressions.

Passing QA

Created attachment 118737 [details] [review]
Bug 15986: (QA follow-up) Fix spelling mistake

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Created attachment 119277 [details] [review]
Bug 15986: (follow-up) Fix usage statement

1.
1824     my $admin_email_address = $library->{branchemail} || C4::Context->preference('KohaAdminEmailAddress');

Shouldn't it be $library->inbound_email_address instead?

2.
1844         next if ($mtt eq 'phone' and C4::Context->preference('TalkingTechItivaPhoneNotification') );

There are now 2 mtt, phone and itiva. Here you need to compare with 'itiva', and add a new condition for 'phone'.

3. Shouldn't we rename "send_notice" to "enqueue_notice" as we don't actually send it right away?

4. Why don't we insert a sample notice template and use it as the default for the script?

5.
+    # If passed message transports we force use those, otherwise we will use the patrons preferences
+    # for the 'Hold_Filled' notice
+    my $sending_params = @mtts ? { message_transports => \@mtts } : { message_name => "Hold_Filled" };

We should add that to the POD

6.
+        # Skip if we already dealt with this borrower
+        next if ( $done{$patron->borrowernumber} );

What if a patron has more than one hold? We won't be able to send a notice like "You have X holds waiting"

7. (not blocker) please don't forget to perltidy new files and blocks of code.

Created attachment 119308 [details] [review]
Bug 15986: (follow-up) Use inbound_email_address for library

(In reply to Jonathan Druart from comment #54)
> 1. Shouldn't it be $library->inbound_email_address instead?

Done

> 2.There are now 2 mtt, phone and itiva. Here you need to compare with 'itiva',
> and add a new condition for 'phone'. 
> 3. Shouldn't we rename "send_notice" to "enqueue_notice" as we don't
> actually send it right away?
> 5. # If passed message transports we force use those, otherwise we will
> use the patrons preferences

These are all done in followups



> 4. Why don't we insert a sample notice template and use it as the default
> for the script?

This can be done on a new bug 

> 6. What if a patron has more than one hold? We won't be able to send a notice
> like "You have X holds waiting"

We can access all of their holds in template toolkit

[% FOREACH hold IN hold.patron.holds %]
...
[% END %]

(In reply to Nick Clemens from comment #55)
> Created attachment 119308 [details] [review] [review]
> Bug 15986: (follow-up) Use inbound_email_address for library

Damn, Shame on me for missing that one..

I agree the others should already be fixed by my followups.

A sample notice would be nice, I agree here with Jonathan really... Especially given the chance sample here where you iterate through holds to construct a digest.

(In reply to Martin Renvoize from comment #57)
> (In reply to Nick Clemens from comment #55)
> > Created attachment 119308 [details] [review] [review] [review]
> > Bug 15986: (follow-up) Use inbound_email_address for library
> 
> Damn, Shame on me for missing that one..
> 
> I agree the others should already be fixed by my followups.

Sorry about that, I usually review all the patches squashed, it seems that I forgot to squash those before review!

> A sample notice would be nice, I agree here with Jonathan really...
> Especially given the chance sample here where you iterate through holds to
> construct a digest.

And shouldn't we actually reflect that in the script? We should loop over the patrons, not the holds.

Created attachment 119327 [details] [review]
Bug 15986: (follow-up) Add all holds per patron

This changes the code to loop through all the holds and group by patron,
we then send the holds to the letter using the 'loops' option

Created attachment 119328 [details] [review]
Bug 15986: Add sample notice and use as default

Created attachment 119329 [details] [review]
Bug 15986: Add sample notice to translated notices

Created attachment 119383 [details] [review]
Bug 15986: (follow-up) Add atomic update

Created attachment 119384 [details] [review]
Bug 15986: Use KohaDates filter to correctly format the waitingdate

Created attachment 119386 [details] [review]
Bug 15986: Fix typo following vs follwing

Few more questions, (sorry!)

1. 
 58    -n                             No email will be sent
shouldn't be replaced with a --confirm flag?

2. Shouldn't we add a new message preference to let patrons a choice to opt-out if they wish? That could be done on a separate bug report but needed for 21.05

Created attachment 119624 [details] [review]
Bug 15986: (follow-up) Switch nomail to confirm

Created attachment 119725 [details] [review]
Bug 15986: move holds_reminder.pl to misc/cronjobs/holds/

Pushed to master for 21.05, thanks to everybody involved!

Created attachment 119748 [details] [review]
Bug 15986: Add missing notice template for it-IT

Follow-up pushed to master.

Enhancement not pushed to 20.11.x

Created attachment 120224 [details] [review]
Bug 15986: Remove \r from HOLD_REMINDER in the yml

To prevent the following errors on `gulp po:update`

misc/translator/Koha-installer.pot:952: warning: internationalized messages should not contain the '\r' escape sequence

Follow-up pushed to master.

Enhancement not pushed to 20.11.x