Bug 36606

Remedies.

These two options are not mutually exclusive.  They are each additive.

An intermediate mail server running free and open source software under direct Koha community control is important for reducing dependence upon and risk of interruption of service from any particular proprietary SMTP service enforcement of policy against excess bounce messages at particular time etc.  Some external SMTP service dedicated to ensuring good email reputation is important in 2024 even if it would be unavoidably proprietary and there may be better choices than secure-mailgate.com.


1. Intermediate Mail Server providing Mail Relay to an External STMP Server.

We can resolve the email authentication problem by sending messages first to a koha-community.org mail server with the primary purpose of providing authentication for email messages before relaying them to secure-mailgate which sends messages to their destination with better  sender reputation than we would have ourselves.

We discussed the issue at the last developer's meeting where I reported that I had investigated the problem and had a DKIM signing mail relay working with secure-mailgate.com with a test server using essentially Postfix running using DockerMailserver.  Testing shows that delivery to Gmail passes all authentication tests for messages originating from the test03.agogme.com and then relaying them via secure-mailgate.com .  I had set the DNS for secure-mailgate.com in same manner as the request for koha-community.org DNS changes with the exception that the sending mailserver m1.test03.agogme.com is set as the MX record for text03.agogme.com which primarily affects receiving email messages, not sending them.      

Choice of Docker Mailserver was the lightest weight choice for providing a Postfix relay service with confidence of ongoing development and bug fixes.  See https://github.com/docker-mailserver/docker-mailserver .  Aiding email collection for email based posting to Discourse with the inclusion of Dovecot and in Docker Mailserver is an added benefit.  Antispam software such as Rspamd, SpamAssassin, etc. may be helpful in reducing the burden of email compatible use of Discourse.  Unneeded software components are easily configured to disable what is unnecessary to reduce the resource use and it would be easy to add components to the Debian based system.  In Docker Mailserver, there is no database use by default to reduce unnecessary resource use and greatly aid debugging.  Consequently, a full service email system with many  

I first tested predominately Postfix Docker services including two good single person projects.  There is no apparent bug list for Alpine GNU/Linux based docker-posfix, https://github.com/hmriit/docker-postfix-dkim .  Postfix Relay also seemed too small for ongoing development such as testing for Debian 12, https://github.com/wader/postfix-relay/ .

If we would need a full mail system running in a Docker container with many senders, a database based system such as Mailcow would be better, https://github.com/mailcow/mailcow-dockerized .  

[Mail servers may benefit more than some other systems from running directly on the host system, even in a chroot gaol, and not in a container for reliability.  Configuration portability advantage of mail systems running in a container may be questionable even when they are full featured systems allowing arbitrary addition and modification.  Beware that Docker services can easily become a black box which reports no helpful diagnostic information about a configuration typing error when failing and restarting every two seconds.  

I have successfully tested configuring Postfix inside docker-mailserver to support sender dependent relaying of mail including relaying via SMTPS port 465 for secure-mailgate.com without preventing other designated senders from sending mail directly without mail relays.


2.  Alternative STMP Service.

Further discussion of the issue at the Hackfest in Marseilles is leading people at HKS3 to explore a possible alternative to secure-mailgate.com which provides more services generally.  Many SMTP services provide DKIM message signing and hosting authentication certificates generally available for secure-mailgate.com .

However, as a free software project and in the interest of being in control of systems and not subject to termination or suspension of services which cannot be replaced quickly enough we should be aiming to reduce or eliminate dependency upon proprietary services.  Particularly with implementing Discourse, there are scenarios such as email addresses used by spammers for unsatisfied account registration; sending an essential message to all users; etc. which may result in temporary excess bounces leading to account suspension or cancellation with a proprietary SMTP providers.

About Secure Mailgate.

There is no necessary problem with secure-mailgate.com , which is performing the valuable function of maintaining mail sender reputation as the ultimate sender for koha-community.org .  Maintaining mail server reputation in 2024 can easily be a full time task which the Koha Community does not have enough resources to provide for ourselves alone.  Reasons such a service is needed include the usual problems for maintaining mailserver reputation and others such as common mail server default configurations which will reject mail from hosting providers because of a problem from an unrelated system in the subnet, lack of DNS resolution for some hosting company virtual domain which appears in email headers despite proper originating domain information, etc.


Part of How Secure Mailgate Works.

Secure Mailgate sends messages from a large number of different subdomains with a large number of different IP addresses.  Any problems which might develop temporarily for part of their system can be worked around by routing to a different subdomain with a different IP address or even a different subnet.

DNS Changes needed for koha-community.org to allow email authentication remedies to work.


1.  DNS SPF Record Update for koha-community.org .

The koha-community.org SPF record needs updating to include the IP address, 75.119.140.119  for hosting an intermediate mailserver sender providing DKIM signatures for automatically generated email on the same system which runs services such as jenkins.  The intermediate mailserver can then relay email messages to secure-mailgate.com, any possible replacement of secure-mailgate.com, and provide continuity for email in case of service interruption or discontinuation from secure-mailgate.com or any future replacement.


1.1.  DNS SPF Record in BIND Format.

In BIND format the changed the updated SPF record would be:

koha-community.org.  IN  TXT  "v=spf1 ip4:75.119.140.119 a mx include:secure-mailgate.com ~all"

Note leading trailing dot after the domain.


2.  DNS A Record for Intermediate Mailserver and Relay Sender Subdomain for koha-community.org 

A subdomain DNS A record for the mailserver functioning as a relay sender is needed.  An MX record is not needed as there is an MX record for the service is running at hetzner.trust-box.at .


2.1.  DNS A Record in BIND Format.

In BIND format the additional line would be:

m01  IN A  75.119.140.119

It will be discovered in due course and be the direct recipient of some junk but best not to name it mail and make it an extra obvious target.  A distinct subdomain is needed to avoid looping behaviour from the mailserver.


3.  Reverse DNS PTR Record for Mailserver and Relay Sender Subdomain in koha-community.org .

Reverse DNS records are very important for mailservers to avoid denylisting but perhaps you have a system which automatically creates reverse DNS PTR records for DNS A records.


3.1.  Reverse DNS PTR Record in BIND Format.

In BIND format the reverse DNS additional PTR record could be as follows but the configuration of the reverse DNS zone would need to be checked for possible leading quad, 119 in this possible case. 

119  IN PTR  m01.koha-community.org.

The leading quad could be empty depending on the configuration of the reverse DNS zone for 75.119.140.119 .

Note the trailing dot after the domain.


4.  DNS DKIM Public Key Record for koha-community.org .

I created a 2048 bit RSA DKIM key selector mail-koha .


4.1  DNS DKIM Public Key Record in BIND Format.

In BIND format, the mail-koha DKIM public key in DNS may be formatted and added as follows. 

mail-koha._domainkey IN  TXT ( "v=DKIM1; g=*; k=rsa; p="
    "pMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAyylPuDr3VwlRB"
    "eGypamhx97/7f7watPZl4z765mU5GfV5S/HSs5oW5E9QK6h/lCG+jHTSiE"
    "d+NG6SzkNaSjHY5dRZyXUWLJb9FMo/0xhACjkZdVvL+g8TYdx961TsrdKQ"
    "aG1tYJ3q3NnrowZQlDALQtk+SLPFYhvbr8kptwskGTnJ2VTh+Bc6kqVgT2"
    "lIwMiy9axMktGZkVrByrCf8KTLCWZ2VtJzz/0DE4vbC3uMC1n3ofFjrdxD"
    "mHg3Hke+X063e45u6Z9p597MnYpIa3C3JqkeR7/dAgxY5X6ZJrhIsXvjbd"
    "pYBzeINPLAVzxmi8NX61pb7yrLZ9FKxqv4++FBQIDAQAB" )

BIND allows a maximum of 255 characters per line.  Reportedly, all the parts could go on one line in BIND with the quote separation of segments as long as segments do not exceed 255 characters but if not officially supported some future BIND record parser may break the one line workaround. 

Note the dot and then underscore after the DKIM key selector.


5.  DMARC Record for koha-community.org .

There is no DMARC record for koha-commnuity.org .

5.1.  DMARC Record in BIND Format.

In BIND format a DMARC record would be:

_dmarc.koha-community.org.  IN  TXT "v=DMARC1; p=none"

Note leading underscore before dmarc and trailing dot after domain.

p=none effectively means no rejection policy set and ensures maximum delivery for email sent from koha-community.org .

At the beginning of the month, I identified a mistake where I had accidentally left a leading character "p" in manually reformatting the public key for easy sending via email.  We had not identified this mistake quickly because a sufficient time window was still needed to add a properly tested Docker Mailserver configuration to the system where portainer.koha-community.org runs. Chris Cormack updated the DNS to remove the leading "p".

Today, Tomás Cohen Arazi has added the Docker Mailserver configuration to the portainer.koha-community.org system which I had tested for DKIM signing and then relaying automated messages from the wiki, jenkins, etc. via secure-mailgate.com or any successor service.  There would also be the option of sending messages directly without the relay if needed with a simple change in the configuration.

The hosting provider Contabo seems to be blocking mailserver ports at least for outgoing messages as is common to prevent spammers from quickly registering accounts and ruining the reputation of the service.  Tomás will need to ask to have the ports unblocked.