Bug 24588

Summary: Set `Mailer-X` and `MessageID` mail headers to reduce the likelihood of Koha mail being marked as spam
Product: Koha Reporter: Ulrich Kleiber <ulrich.kleiber>
Component: NoticesAssignee: Ulrich Kleiber <ulrich.kleiber>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: 1joynelson, jonathan.druart, katrin.fischer, martin.renvoize
Version: MainKeywords: dependency
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.05.00
Attachments: Bug 24588: Prevent e-mails from Koha to be flagged as spam
Bug 24588: Prevent e-mails from Koha to be flagged as spam
Bug 24588: Prevent e-mails from Koha to be flagged as spam
Bug 24588: Prevent e-mails from Koha to be flagged as spam
Bug 24588: Prevent e-mails from Koha to be flagged as spam
Bug 24588: Prevent e-mails from Koha to be flagged as spam

Description Ulrich Kleiber 2020-02-05 08:07:11 UTC
Set the e-mail headers Mailer-X und MessageID by Koha,
to gain a better positive score from the spam filter.
Comment 1 Ulrich Kleiber 2020-02-05 08:41:28 UTC
Created attachment 98430 [details] [review]
Bug 24588: Prevent e-mails from Koha to be flagged as spam
Comment 2 Ulrich Kleiber 2020-02-05 10:08:23 UTC
Created attachment 98441 [details] [review]
Bug 24588: Prevent e-mails from Koha to be flagged as spam
Comment 3 Owen Leonard 2020-02-05 13:08:15 UTC
Please add a commit message with a test plan:

https://wiki.koha-community.org/wiki/Commit_messages
Comment 4 Ulrich Kleiber 2020-02-05 14:06:24 UTC
Created attachment 98484 [details] [review]
Bug 24588: Prevent e-mails from Koha to be flagged as spam

Set the e-mail headers Mailer-X and MessageID by Koha,
to gain a better positive score from the spam filter.

To test:
1) Let Koha send an e-mail message.
2) Check the header of this e-mail message with your e-mail client.
   There shoud be no X-Mailer header with value 'Koha'. If there
   is an MessageID header, it is probably set by the MSA, e.g. Exim.
   Example:
      Message-Id: <E1iyZRY-0006XT-0t@FQDN>
3) Apply patch.
4) Repeat step 1.
5) Check the header of the new e-mail message with your e-mail client.
   There shoud be a X-Mailer header with value 'Koha' and a MessageID
   header with a value set by Koha.
   Example:
      X-Mailer: Koha
      Message-Id: <15807387030.B731e.18490@FQDN>
6) Sign off.
Comment 5 Ulrich Kleiber 2020-02-05 14:10:34 UTC
Created attachment 98489 [details] [review]
Bug 24588: Prevent e-mails from Koha to be flagged as spam

Set the e-mail headers Mailer-X and MessageID by Koha,
to gain a better positive score from the spam filter.

To test:
1) Let Koha send an e-mail message.
2) Check the header of this e-mail message with your e-mail client.
   There should be no X-Mailer header with value 'Koha'. If there
   is an MessageID header, it is probably set by the MSA, e.g. Exim.
   Example:
      Message-Id: <E1iyZRY-0006XT-0t@FQDN>
3) Apply patch.
4) Repeat step 1.
5) Check the header of the new e-mail message with your e-mail client.
   There should be a X-Mailer header with value 'Koha' and a MessageID
   header with a value set by Koha.
   Example:
      X-Mailer: Koha
      Message-Id: <15807387030.B731e.18490@FQDN>
6) Sign off.
Comment 6 Martin Renvoize 2020-02-05 14:55:32 UTC
Thanks Ulrich :)
Comment 7 Martin Renvoize 2020-02-06 09:00:54 UTC
Created attachment 98518 [details] [review]
Bug 24588: Prevent e-mails from Koha to be flagged as spam

Set the e-mail headers Mailer-X and MessageID by Koha,
to gain a better positive score from the spam filter.

To test:
1) Let Koha send an e-mail message.
2) Check the header of this e-mail message with your e-mail client.
   There should be no X-Mailer header with value 'Koha'. If there
   is an MessageID header, it is probably set by the MSA, e.g. Exim.
   Example:
      Message-Id: <E1iyZRY-0006XT-0t@FQDN>
3) Apply patch.
4) Repeat step 1.
5) Check the header of the new e-mail message with your e-mail client.
   There should be a X-Mailer header with value 'Koha' and a MessageID
   header with a value set by Koha.
   Example:
      X-Mailer: Koha
      Message-Id: <15807387030.B731e.18490@FQDN>
6) Sign off.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize 2020-02-06 09:01:53 UTC
Works as expected, no regressions found and it moves us closer to the ever-moving target that is 'email best practice'.

Signing off
Comment 9 Ulrich Kleiber 2020-02-06 10:23:13 UTC
Thanks Martin :)
Comment 10 Jonathan Druart 2020-02-21 12:30:16 UTC
1. Should not we want the X-Mailer value to be more specific? Like with the library name for instance?


2. I suspect Email::MessageID to be "buggy" under plack.

My understanding (may be wrong!):

"my %uniq" is supposed to contain " $epoch => $nb_occ " with $nb_occ the number of time $epoch has been associated to an id.

Say the following Message-ID generated by the module:
15822877730.63BC8.7268@df2c0b4bf86c
1582287773 is $epoch, 0 is the first occurrence
63BC8 is a random value
7268@df2c0b4bf86c is the hostname

The next ID generated in the same second will be 15822877731.
If another script is ran at the same time, we will have this "uniq" 15822877730 used several time.


Short version: All of that to say that we are safe as there is the random value that is concatenated, it will be bad luck to get twice the same generated random value, in the same second, with 2 scripts . But it is possible :)
Comment 11 Jonathan Druart 2020-02-21 12:32:51 UTC
(In reply to Jonathan Druart from comment #10)

You can really forget 2. as the same behavior exists without Plack (the variable will obviously not be shared). I guess that's why the author added the randomly generated string.
Comment 12 Katrin Fischer 2020-02-21 14:17:04 UTC
I found this explanation:
"the X-Mailer line in the email header tells you what program was used to draft and send the original email"

Example:
X-Mailer: Mozilla 4.05 [de] (Win95; I)

Koha seems accurate enough.
Comment 13 Jonathan Druart 2020-02-21 14:26:57 UTC
Created attachment 99384 [details] [review]
Bug 24588: Prevent e-mails from Koha to be flagged as spam

Set the e-mail headers Mailer-X and MessageID by Koha,
to gain a better positive score from the spam filter.

To test:
1) Let Koha send an e-mail message.
2) Check the header of this e-mail message with your e-mail client.
   There should be no X-Mailer header with value 'Koha'. If there
   is an MessageID header, it is probably set by the MSA, e.g. Exim.
   Example:
      Message-Id: <E1iyZRY-0006XT-0t@FQDN>
3) Apply patch.
4) Repeat step 1.
5) Check the header of the new e-mail message with your e-mail client.
   There should be a X-Mailer header with value 'Koha' and a MessageID
   header with a value set by Koha.
   Example:
      X-Mailer: Koha
      Message-Id: <15807387030.B731e.18490@FQDN>
6) Sign off.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 14 Martin Renvoize 2020-02-21 16:08:29 UTC
Nice work everyone!

Pushed to master for 20.05
Comment 15 Joy Nelson 2020-03-13 00:54:35 UTC
enhancement not backported to 19.11.x