Bug 14590

Summary: Validate messaging preferences
Product: Koha Reporter: Lari Taskula <lari.taskula>
Component: PatronsAssignee: Lari Taskula <lari.taskula>
Status: BLOCKED --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: george, gmcharlt, gwilliams, jonathan.druart, kyle.m.hall, severine.queune, veron
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31424
Change sponsored?: --- Patch complexity: Medium patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Bug Depends on: 14620, 5685, 14621    
Bug Blocks:    
Attachments: Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
Bug 14590 - PageObject tests for misconfigured messaging preferences
Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
[Signed-off] Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences
Bug 14590: Validate messaging preferences
Bug 14590: Validate messaging preferences
Bug 14590: Validate messaging preferences
Bug 14590: Validate messaging preferences

Description Lari Taskula 2015-07-22 13:45:05 UTC
Currently, in Patron message preferences, it is possible to set message transport type without valid contact information (e.g. send sms even if patron does not have a phone number set).

This is very problematic for us since we have several Borrowers with misconfigured messaging preferences, and that causes issues, like Borrowers not knowing they have Items waiting for pickup. This makes customers angry and our librarians sad, not to mention the technical staff.
Reasons for this are myriad but one big challenge is misconfigured messaging preferences, both in OPAC and the staff client.

I have listed messaging preferences that would need to require valid contact information:
#1 SMS (this requires valid SMSnumber)
#2 Phone (this requires valid phone)
#3 Email (this requires valid email)

Validations will be done both client-side and server-side. I am working on the following plan:

Server-side validation 
- In C4::Form::MessagingPreferences::handle_form_action(), check for each parameter (SMSnumber, phone, email) to see if they are provided.
- If the parameter is provided, allow message_transport_types to contain said transport method for this message.
- If the parameter is not provided, remove the said transport method for this message from message_transport_types.
 
Client-side validation
- Has to be done in both OPAC and staff client.
- With JavaScript, check the contents of each contact field (SMSnumber, phone, email).
- If the contact field is invalid or empty, remove all checks from messaging preferences checkboxes for this transport type. Also disable these checkboxes.
- Once the contact field becomes valid on input event, enable checkboxes for this transport type and restore any previously stored state (e.g. we just want to change phone number without setting all messaging preferences again).
Comment 1 Lari Taskula 2015-08-14 12:18:23 UTC Comment hidden (obsolete)
Comment 2 Lari Taskula 2015-08-14 12:19:06 UTC Comment hidden (obsolete)
Comment 3 Lari Taskula 2015-08-17 08:05:39 UTC Comment hidden (obsolete)
Comment 4 Lari Taskula 2015-08-17 10:45:10 UTC Comment hidden (obsolete)
Comment 5 Lari Taskula 2015-08-20 08:45:33 UTC Comment hidden (obsolete)
Comment 6 Lari Taskula 2015-08-20 08:48:14 UTC Comment hidden (obsolete)
Comment 7 Lari Taskula 2015-08-25 07:21:43 UTC Comment hidden (obsolete)
Comment 8 Lari Taskula 2015-08-26 14:15:28 UTC Comment hidden (obsolete)
Comment 9 Lari Taskula 2015-08-28 14:07:57 UTC
This patch makes both client and server side validations, both in OPAC and Staff client.

Depends on Bug 14620.

Test plan for Staff client:
1. Login to Staff client
2. Enable syspref ValidateEmailAddress and set ValidatePhoneNumber to "ipn"
3. Enable syspref TalkingTechItivaPhoneNotification and SMSSendDriver (write anything), and allow syspref EnhancedMessagingPreferences 
4. Navigate to modify patron's information at memberentry.pl
5. Insert valid primary phone number, primary email address and SMS number
6. Check all messaging preferences and submit changes
7. Navigate to modify patron's information at memberentry.pl
8. Clear primary email, primary phone and SMS number fields and submit changes
9. Observe that the messaging preferences are disabled

Test plan for OPAC:
-1. Make sure sysprefs are set like in Staff client test step 2-3
1. Login to OPAC
2. Navigate to "your personal details"
3. Insert valid primary phone number, primary email address and SMS number
4. Submit changes
5. Login to Staff client
5.1. Navigate to Patrons
5.2. Approve information modification request
6. Back in OPAC, Navigate to "your messaging"
7. Check all messaging preferences and submit changes
8. Navigate to "your personal details"
9. Clear primary email, primary phone and submit changes
10. Repeat step 5 and 6
11. Observe that the messaging preferences are disabled for "Phone" and "Email" columns
12. Clear SMS number field and submit changes
13. Observe that all the messaging preferences are disabled

This patch also includes a maintenance script at misc/maintenance/deleteMisconfiguredMessagingPrefs.pl that goes through all borrowers 
and automatically deletes their misconfigured messaging preferences.
Comment 10 Lari Taskula 2015-09-08 13:47:56 UTC Comment hidden (obsolete)
Comment 11 Marc Véron 2015-09-09 13:11:35 UTC Comment hidden (obsolete)
Comment 12 Jonathan Druart 2015-09-15 08:23:44 UTC
Blocked by bug 14620 which is blocked by bug 7174.
Comment 13 Lari Taskula 2015-11-10 14:22:12 UTC
Created attachment 44704 [details] [review]
Bug 14590 - Checkboxes should be disabled without valid contact information in messaging preferences

This patch makes both client and server side validations, both in OPAC and Staff client.

Test plan for Staff client:
1. Login to Staff client
2. Enable syspref ValidateEmailAddress and set ValidatePhoneNumber to "ipn"
3. Enable syspref TalkingTechItivaPhoneNotification and SMSSendDriver (write anything), and allow syspref EnhancedMessagingPreferences
4. Navigate to modify patron's information at memberentry.pl
5. Insert valid primary phone number, primary email address and SMS number
6. Check all messaging preferences and submit changes
7. Navigate to modify patron's information at memberentry.pl
8. Clear primary email, primary phone and SMS number fields and submit changes
9. Observe that the messaging preferences are disabled

Test plan for OPAC:
-1. Make sure sysprefs are set like in Staff client test step 2-3
1. Login to OPAC
2. Navigate to "your personal details"
3. Insert valid primary phone number, primary email address and SMS number
4. Submit changes
5. Login to Staff client
5.1. Navigate to Patrons
5.2. Approve information modification request
6. Back in OPAC, Navigate to "your messaging"
7. Check all messaging preferences and submit changes
8. Navigate to "your personal details"
9. Clear primary email, primary phone and submit changes
10. Repeat step 5 and 6
11. Observe that the messaging preferences are disabled for "Phone" and "Email" columns
12. Clear SMS number field and submit changes
13. Observe that all the messaging preferences are disabled

This patch also includes a maintenance script at misc/maintenance/deleteMisconfiguredMessagingPrefs.pl that goes through all borrowers
and automatically deletes their misconfigured messaging preferences.

Fixing category messaging preferences broken by previous patch.
Comment 14 Lari Taskula 2017-04-11 12:32:59 UTC
Created attachment 62040 [details] [review]
Bug 14590: Validate messaging preferences

Messaging preferences should be only available if a valid contact information
is stored for the patron for that transport type. Otherwise, it should not be
possible to select preferences for that transport type.

For example, when attempting to set messages to be delivered via sms, we should
automatically check patron's smsalertnumber before allowing this type of
modification. If patron does not have a valid smsalertnumber and is not passing
it along with the messaging preference change, we should not store those given
messaging transport types.

This patch implements exactly this. It uses Koha::Validation from Bug 14620
to validate contact informations and attaches it into patron modification
logic - whenever messaging preferences are processed by Koha, preferences
that link to an invalid or non-existent contact information will be removed.

Client-side JavaScript checks are also implemented. When modifying patron
information, messaging preferences for a certain transport type will be either
enabled and/or disabled on the fly depending on validity of the contact info
linked to that transport type (transport type "sms" => borrower "smsalertnumber").

TESTING:
This feature requires tests both in OPAC and staff client.

Requirements:
1. Apply patch(es) in Bug 14620 before this patch
2. Make sure EnhancedMessagingPreferences is enabled
3. Make sure EnhancedMessagingPreferencesOPAC is enabled
4. Make sure TalkingTechItivaPhoneNotification is enabled (activates transport
   type "phone")
5. Make sure SMSSendDriver is enabled (for simple testing purposes you can use
   the value "test")
6. Preference "ValidatePhoneNumber" needs a regular expression for validating
   phone numbers. An example regular expression is provided below the input
   field. You can use that while testing this feature. Copy-paste it into the
   input box and save preferences. (regex also provided below)
^((\+)?[1-9]{1,2})?([-\s\.])?((\(\d{1,4}\))|\d{1,4})(([-\s\.])?[0-9]{1,12}){1,2}$

TEST PLAN:
1. Login to staff client
2. Go to modification view of any patron
3. Scroll down all the way and you should see "Patron messaging preferences"
   table including checkboxes for transport types "SMS", "Phone" and "Email".
4. If the patron you are modifying has any contact information of the following
   "Primary email", "Primary phone", "SMS number", please clear those fields.
5. Scroll down. All checkboxes for the given transport types should now
   be disabled.
6. Modify field "Primary email" and insert "test". A warning of invalid email
   should appear.
7. Scroll down. Checkboxes for transport type "Email" should still be disabled.
8. Modify field "Primary email" and insert "nobody@example.com".
9. Scroll down. Checkboxes for transport type "Email" should now be enabled.
10. Select email as transport type for each message type and click "Save".
11. Go back to modification screen. The messaging preferences should still
    appear as you stored them.
12. Repeat steps 6-11 also for both "Phone" and "SMS number". When inserting
    text into phone number fields, they should appear as invalid, so test
    like that.
13. After successful testing, test also other patron modification screens. Go
    to patron's Details tab and you should see multiple "Edit" links on that
    page. The point is to make sure transport type selections are correctly
    removed from the patron if the modification screen lets you modify for
    example primary email, but does not present the messaging preferences
    at the bottom of the page. So, as an example, first insert patron a valid
    email and select all email transport types. Then, go to the "Edit" link
    right above "Patron messaging preferences" title on patron's details page.
    This page lets you modify "Primary email" but not messaging preferences.
    Clear the "Primary email" field and save, and observe that all previously
    selected email transport types are now removed.
14. Give your patron a valid email and go to Details tab -> "Edit" link below
    "Patron messaging preferences title" on the left. You should still be able
    to select email as transport types.
15. Go remove patron's primary email and repeat step 14. You will observe that
    it is no longer possible to select email as a transport type.
16. Try steps 13-15 with other transport types as well

After this test plan you should have a pretty good idea of the feature. As
mentioned earlier, this feature requires tests on the OPAC side as well. So
attempt this also via OPAC and make sure preferences are validated when the
librarian approves patron's modifications.
Comment 15 Lari Taskula 2017-04-11 13:35:18 UTC
Created attachment 62043 [details] [review]
Bug 14590: Validate messaging preferences

Messaging preferences should be only available if a valid contact information
is stored for the patron for that transport type. Otherwise, it should not be
possible to select preferences for that transport type.

For example, when attempting to set messages to be delivered via sms, we should
automatically check patron's smsalertnumber before allowing this type of
modification. If patron does not have a valid smsalertnumber and is not passing
it along with the messaging preference change, we should not store those given
messaging transport types.

This patch implements exactly this. It uses Koha::Validation from Bug 14620
to validate contact informations and attaches it into patron modification
logic - whenever messaging preferences are processed by Koha, preferences
that link to an invalid or non-existent contact information will be removed.

Client-side JavaScript checks are also implemented. When modifying patron
information, messaging preferences for a certain transport type will be either
enabled and/or disabled on the fly depending on validity of the contact info
linked to that transport type (transport type "sms" => borrower "smsalertnumber").

TESTING:
This feature requires tests both in OPAC and staff client.

Requirements:
1. Apply patch(es) in Bug 14620 before this patch
2. Make sure EnhancedMessagingPreferences is enabled
3. Make sure EnhancedMessagingPreferencesOPAC is enabled
4. Make sure TalkingTechItivaPhoneNotification is enabled (activates transport
   type "phone")
5. Make sure SMSSendDriver is enabled (for simple testing purposes you can use
   the value "test")
6. Preference "ValidatePhoneNumber" needs a regular expression for validating
   phone numbers. An example regular expression is provided below the input
   field. You can use that while testing this feature. Copy-paste it into the
   input box and save preferences. (regex also provided below)
^((\+)?[1-9]{1,2})?([-\s\.])?((\(\d{1,4}\))|\d{1,4})(([-\s\.])?[0-9]{1,12}){1,2}$

TEST PLAN:
1. Login to staff client
2. Go to modification view of any patron
3. Scroll down all the way and you should see "Patron messaging preferences"
   table including checkboxes for transport types "SMS", "Phone" and "Email".
4. If the patron you are modifying has any contact information of the following
   "Primary email", "Primary phone", "SMS number", please clear those fields.
5. Scroll down. All checkboxes for the given transport types should now
   be disabled.
6. Modify field "Primary email" and insert "test". A warning of invalid email
   should appear.
7. Scroll down. Checkboxes for transport type "Email" should still be disabled.
8. Modify field "Primary email" and insert "nobody@example.com".
9. Scroll down. Checkboxes for transport type "Email" should now be enabled.
10. Select email as transport type for each message type and click "Save".
11. Go back to modification screen. The messaging preferences should still
    appear as you stored them.
12. Repeat steps 6-11 also for both "Phone" and "SMS number". When inserting
    text into phone number fields, they should appear as invalid, so test
    like that.
13. After successful testing, test also other patron modification screens. Go
    to patron's Details tab and you should see multiple "Edit" links on that
    page. The point is to make sure transport type selections are correctly
    removed from the patron if the modification screen lets you modify for
    example primary email, but does not present the messaging preferences
    at the bottom of the page. So, as an example, first insert patron a valid
    email and select all email transport types. Then, go to the "Edit" link
    right above "Patron messaging preferences" title on patron's details page.
    This page lets you modify "Primary email" but not messaging preferences.
    Clear the "Primary email" field and save, and observe that all previously
    selected email transport types are now removed.
14. Give your patron a valid email and go to Details tab -> "Edit" link below
    "Patron messaging preferences title" on the left. You should still be able
    to select email as transport types.
15. Go remove patron's primary email and repeat step 14. You will observe that
    it is no longer possible to select email as a transport type.
16. Try steps 13-15 with other transport types as well

After this test plan you should have a pretty good idea of the feature. As
mentioned earlier, this feature requires tests on the OPAC side as well. So
attempt this also via OPAC and make sure preferences are validated when the
librarian approves patron's modifications.
Comment 16 Lari Taskula 2017-04-11 13:49:25 UTC
Rebased on master & rewrote much of the patch and added a more detailed test plan.
Comment 17 Lari Taskula 2017-04-11 13:54:09 UTC
Created attachment 62044 [details] [review]
Bug 14590: Validate messaging preferences

Messaging preferences should be only available if a valid contact information
is stored for the patron for that transport type. Otherwise, it should not be
possible to select preferences for that transport type.

For example, when attempting to set messages to be delivered via sms, we should
automatically check patron's smsalertnumber before allowing this type of
modification. If patron does not have a valid smsalertnumber and is not passing
it along with the messaging preference change, we should not store those given
messaging transport types.

This patch implements exactly this. It uses Koha::Validation from Bug 14620
to validate contact informations and attaches it into patron modification
logic - whenever messaging preferences are processed by Koha, preferences
that link to an invalid or non-existent contact information will be removed.

Client-side JavaScript checks are also implemented. When modifying patron
information, messaging preferences for a certain transport type will be either
enabled and/or disabled on the fly depending on validity of the contact info
linked to that transport type (transport type "sms" => borrower "smsalertnumber").

TESTING:
This feature requires tests both in OPAC and staff client.

Requirements:
1. Apply patch(es) in Bug 14620 before this patch
2. Make sure EnhancedMessagingPreferences is enabled
3. Make sure EnhancedMessagingPreferencesOPAC is enabled
4. Make sure TalkingTechItivaPhoneNotification is enabled (activates transport
   type "phone")
5. Make sure SMSSendDriver is enabled (for simple testing purposes you can use
   the value "test")
6. Preference "ValidatePhoneNumber" needs a regular expression for validating
   phone numbers. An example regular expression is provided below the input
   field. You can use that while testing this feature. Copy-paste it into the
   input box and save preferences. (regex also provided below)
^((\+)?[1-9]{1,2})?([-\s\.])?((\(\d{1,4}\))|\d{1,4})(([-\s\.])?[0-9]{1,12}){1,2}$

TEST PLAN:
1. Login to staff client
2. Go to modification view of any patron
3. Scroll down all the way and you should see "Patron messaging preferences"
   table including checkboxes for transport types "SMS", "Phone" and "Email".
4. If the patron you are modifying has any contact information of the following
   "Primary email", "Primary phone", "SMS number", please clear those fields.
5. Scroll down. All checkboxes for the given transport types should now
   be disabled.
6. Modify field "Primary email" and insert "test". A warning of invalid email
   should appear.
7. Scroll down. Checkboxes for transport type "Email" should still be disabled.
8. Modify field "Primary email" and insert "nobody@example.com".
9. Scroll down. Checkboxes for transport type "Email" should now be enabled.
10. Select email as transport type for each message type and click "Save".
11. Go back to modification screen. The messaging preferences should still
    appear as you stored them.
12. Repeat steps 6-11 also for both "Phone" and "SMS number". When inserting
    text into phone number fields, they should appear as invalid, so test
    like that.
13. After successful testing, test also other patron modification screens. Go
    to patron's Details tab and you should see multiple "Edit" links on that
    page. The point is to make sure transport type selections are correctly
    removed from the patron if the modification screen lets you modify for
    example primary email, but does not present the messaging preferences
    at the bottom of the page. So, as an example, first insert patron a valid
    email and select all email transport types. Then, go to the "Edit" link
    right above "Patron messaging preferences" title on patron's details page.
    This page lets you modify "Primary email" but not messaging preferences.
    Clear the "Primary email" field and save, and observe that all previously
    selected email transport types are now removed.
14. Give your patron a valid email and go to Details tab -> "Edit" link below
    "Patron messaging preferences title" on the left. You should still be able
    to select email as transport types.
15. Go remove patron's primary email and repeat step 14. You will observe that
    it is no longer possible to select email as a transport type.
16. Try steps 13-15 with other transport types as well

After this test plan you should have a pretty good idea of the feature. As
mentioned earlier, this feature requires tests on the OPAC side as well. So
attempt this also via OPAC and make sure preferences are validated when the
librarian approves patron's modifications.
Comment 18 Lari Taskula 2017-04-12 08:26:02 UTC
Created attachment 62071 [details] [review]
Bug 14590: Validate messaging preferences

Messaging preferences should be only available if a valid contact information
is stored for the patron for that transport type. Otherwise, it should not be
possible to select preferences for that transport type.

For example, when attempting to set messages to be delivered via sms, we should
automatically check patron's smsalertnumber before allowing this type of
modification. If patron does not have a valid smsalertnumber and is not passing
it along with the messaging preference change, we should not store those given
messaging transport types.

This patch implements exactly this. It uses Koha::Validation from Bug 14620
to validate contact informations and attaches it into patron modification
logic - whenever messaging preferences are processed by Koha, preferences
that link to an invalid or non-existent contact information will be removed.

Client-side JavaScript checks are also implemented. When modifying patron
information, messaging preferences for a certain transport type will be either
enabled and/or disabled on the fly depending on validity of the contact info
linked to that transport type (transport type "sms" => borrower "smsalertnumber").

TESTING:
This feature requires tests both in OPAC and staff client.

Requirements:
1. Apply patch(es) in Bug 14620 before this patch
2. Make sure EnhancedMessagingPreferences is enabled
3. Make sure EnhancedMessagingPreferencesOPAC is enabled
4. Make sure TalkingTechItivaPhoneNotification is enabled (activates transport
   type "phone")
5. Make sure SMSSendDriver is enabled (for simple testing purposes you can use
   the value "test")
6. Preference "ValidatePhoneNumber" needs a regular expression for validating
   phone numbers. An example regular expression is provided below the input
   field. You can use that while testing this feature. Copy-paste it into the
   input box and save preferences. (regex also provided below)
^((\+)?[1-9]{1,2})?([-\s\.])?((\(\d{1,4}\))|\d{1,4})(([-\s\.])?[0-9]{1,12}){1,2}$

TEST PLAN:
1. Login to staff client
2. Go to modification view of any patron
3. Scroll down all the way and you should see "Patron messaging preferences"
   table including checkboxes for transport types "SMS", "Phone" and "Email".
4. If the patron you are modifying has any contact information of the following
   "Primary email", "Primary phone", "SMS number", please clear those fields.
5. Scroll down. All checkboxes for the given transport types should now
   be disabled.
6. Modify field "Primary email" and insert "test". A warning of invalid email
   should appear.
7. Scroll down. Checkboxes for transport type "Email" should still be disabled.
8. Modify field "Primary email" and insert "nobody@example.com".
9. Scroll down. Checkboxes for transport type "Email" should now be enabled.
10. Select email as transport type for each message type and click "Save".
11. Go back to modification screen. The messaging preferences should still
    appear as you stored them.
12. Repeat steps 6-11 also for both "Phone" and "SMS number". When inserting
    text into phone number fields, they should appear as invalid, so test
    like that.
13. After successful testing, test also other patron modification screens. Go
    to patron's Details tab and you should see multiple "Edit" links on that
    page. The point is to make sure transport type selections are correctly
    removed from the patron if the modification screen lets you modify for
    example primary email, but does not present the messaging preferences
    at the bottom of the page. So, as an example, first insert patron a valid
    email and select all email transport types. Then, go to the "Edit" link
    right above "Patron messaging preferences" title on patron's details page.
    This page lets you modify "Primary email" but not messaging preferences.
    Clear the "Primary email" field and save, and observe that all previously
    selected email transport types are now removed.
14. Give your patron a valid email and go to Details tab -> "Edit" link below
    "Patron messaging preferences title" on the left. You should still be able
    to select email as transport types.
15. Go remove patron's primary email and repeat step 14. You will observe that
    it is no longer possible to select email as a transport type.
16. Try steps 13-15 with other transport types as well

After this test plan you should have a pretty good idea of the feature. As
mentioned earlier, this feature requires tests on the OPAC side as well. So
attempt this also via OPAC and make sure preferences are validated when the
librarian approves patron's modifications.
Comment 19 Katrin Fischer 2017-10-07 21:02:26 UTC
I am sorry, this does no longer apply:

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 14590: Validate messaging preferences
.git/rebase-apply/patch:663: trailing whitespace.
	print scalar ($destination eq "circ") ? 
error: sha1 information is lacking or useless (Koha/Validation.pm).
error: could not build fake ancestor
Patch failed at 0001 Bug 14590: Validate messaging preferences
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-14590-Validate-messaging-preferences-JMLoUs.patch


From eyeballing the code I was wondering if you only check 'primary email' and don't take AutoEmailPrimaryAddress into account?
Comment 20 Katrin Fischer 2017-10-07 21:05:20 UTC
Sorry, I had missed the dependency on bug 14620, but this also doesn't apply. Marking this one 'blocked' and the other 'doesn't apply'.