Bug 15303

Summary: Letsencrypt option for Debian package installations
Product: Koha Reporter: Mirko Tietgen <mirko>
Component: PackagingAssignee: Mirko Tietgen <mirko>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: arslanone, black23, dcook, gmcharlt, jonathan.druart+koha, josef.moravec, kholten, kyle, magnus, robin
Version: unspecified   
Hardware: All   
OS: All   
See Also: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8897
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15540
Change sponsored?: --- Patch complexity: ---
Text to go in the release notes:
Version(s) released in:
Bug Depends on:    
Bug Blocks: 15587    
Attachments: Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 --force-https option for koha-create
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Add --letsencrypt to koha-list
Bug 15303 Go back to old version of Apache template
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Use --staging during development to circumvent limits
Bug 15303 LE production server
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Workaround to use git ckeckout
Bug 15303 LE production server
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 Workaround to use git ckeckout
Bug 15303 LE production server
Bug 15303 Followup: apache port 80 servername/alias and typo
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 LE production server
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 LE production server
Bug 15303 Followup for koha-remove
Bug 15303 Followup: Fix apt-cache show breaking koha-create
Bug 15303 QA-Followup: check_letsencrypt only if --letsencrypt is used
Bug 15303 Letsencrypt option for Debian package installations
Bug 15303 LE production server
Bug 15303 Followup for koha-remove
Bug 15303 Followup: Fix apt-cache show breaking koha-create
Bug 15303 QA-Followup: check_letsencrypt only if --letsencrypt is used
Bug 15303: Test if the symlink to letsencrypt-auto exists
Bug 15303 Followup: If install LE dependencies from backports on Jessie
Bug 15303 Followup: If install LE dependencies from backports on Jessie

Description Mirko Tietgen 2015-12-03 23:52:47 UTC
Koha Debian packages should have an option to automatically generate a letsencrypt certificate, validate it and use it in Apache.
Comment 6 Mirko Tietgen 2015-12-04 02:07:26 UTC
The server this is tested on has to be accessible on port 80 from the outside world.
Comment 9 Mirko Tietgen 2015-12-04 06:57:07 UTC
>  --letsencrypt /home/mirko/letsencrypt

Use your own /path/to/letsencryptfolder instead ;)
Comment 13 Mirko Tietgen 2015-12-04 19:48:41 UTC
Looks like i broke it with the last update. Fix coming soon.
Comment 14 Galen Charlton 2015-12-05 06:45:33 UTC
Very promising start. Since the Let's Encrypt certificates are short-lived (90 days, IIRC), there will also need to be a mechanism in place to automatically renew them.  An /etc/cron.monthly/koha-common might do for now (although I'm kinda hoping that the Debian Let's Encrypt packagers come up with a way to centralize management of the periodic cert renewals/replacmenets).
Comment 15 Mirko Tietgen 2015-12-05 10:30:15 UTC
It is supposed to be automated in itself.[1] The experimental Debian package[2] has a letsencrypt and a letencrypt-renew binary.[3]

For now using a copy of the line used to generate the first certificate in a monthly cronjob should work. Or maybe a koha-list --letsencrypt flag that lists all instances created that way and a cronjob that uses this.

I will add something as seperate patch so it can be ommited when its not needed anymore at push time.

[1] https://letsencrypt.readthedocs.org/en/latest/using.html#renewal
[2] https://packages.debian.org/experimental/letsencrypt
[3] https://github.com/letsencrypt/letsencrypt/issues/257
Comment 16 Mirko Tietgen 2015-12-05 12:49:51 UTC
It would probably be better to have a single option for koha-create. Just --letsencrypt <path>, and if path is not given, use default /usr/bin.
Comment 17 Mirko Tietgen 2015-12-05 12:53:32 UTC
The "real" IfDefine version (not on bz) does not work with apache 2.2, this one (using IfDefine as a way to comment out parts of the conf file) does not work for me on 2.4. I will go back to the uglier first version with a lot of # comments in the file that get removed after certs are created.
Comment 18 Mirko Tietgen 2015-12-05 12:54:26 UTC
Setting to in disussion for now, patch does not work atm.
Comment 22 Mirko Tietgen 2015-12-31 11:33:05 UTC
Doing a cronjob for this until automatic renewal is in letsencrypt itself only makes sense if this is going to be pushed to stable releases soon. It will very likely work automatically by the time 3.24 is released.

So I'd like to hear from package maintainers if they would like to push this, before I work on it.
Comment 23 Mirko Tietgen 2015-12-31 11:50:54 UTC
I put some information for testers in the wiki at http://wiki.koha-community.org/wiki/LetsEncrypt

I want to roll a package and test this again after the last changes before I set it back to needs signoff.
Comment 35 Mirko Tietgen 2016-02-27 13:13:49 UTC
letsencrypt is in jessie-backports now https://packages.debian.org/jessie-backports/letsencrypt
Comment 36 Mirko Tietgen 2016-03-23 21:52:16 UTC Comment hidden (obsolete)
Comment 37 Mirko Tietgen 2016-03-23 21:52:23 UTC Comment hidden (obsolete)
Comment 38 Chris Cormack 2016-04-13 21:56:28 UTC Comment hidden (obsolete)
Comment 39 Chris Cormack 2016-04-13 21:56:54 UTC Comment hidden (obsolete)
Comment 40 Mirko Tietgen 2016-04-15 10:38:36 UTC
About renewal, the Debian LE package can do that automatically now. letsencrypt --renew renews all certs that are valid for < 30 days I think. You can test that with

> letsencrypt --renew --dry-run

LE puts the conf files in /etc/letsencrypt/renewal/$opacdomain.conf

I will provide a followup to remove that with koha-remove to avoid problems. I will also remove $instance/letsencrypt.enabled to prevent errors. The folder is not deleted automatically by koha-remove.
Comment 41 Mirko Tietgen 2016-04-15 10:40:52 UTC
It's actually

letsencrypt renew --dry-run
Comment 42 Mirko Tietgen 2016-04-15 13:27:35 UTC Comment hidden (obsolete)
Comment 43 Jonathan Druart 2016-04-15 13:46:47 UTC
I am getting crazy with my setup, I have not managed to test this patch entirely to pass QA on it: cannot package it, cannot install the .deb Mirko gave me (kohadevbox:ansible raises lot of updatedatabase warnings, on a lxc koha-common stuck on random step (configure apache-mpm-itk or cron or at, etc.)
Reading the code, everything looks good.
Hope to see someone QA it before the feature freeze.
Comment 44 Jonathan Druart 2016-04-15 14:48:21 UTC
Using Jessie without backport enabled:

sudo koha-create --create-db --letsencrypt LE
Does not return anything but errno 100

Because
% apt-cache show letsencrypt
  N: Unable to locate package letsencrypt
  E: No packages found
% echo $?
  100

The line
+        apt-cache show letsencrypt &>/dev/null
is wrong
Comment 45 Mirko Tietgen 2016-04-15 16:06:10 UTC Comment hidden (obsolete)
Comment 46 Jonathan Druart 2016-04-15 16:25:39 UTC
After discussion with Mirko, we have found some others issues with the --letsencrypt param:

check_letsencrypt should not be called unless the param is passed
If you are a liar and the symlink does not exist, you get a weird error from adduser (adduser: Please enter a username matching the regular expression configured)
Comment 47 Mirko Tietgen 2016-04-15 16:31:22 UTC Comment hidden (obsolete)
Comment 48 Mirko Tietgen 2016-04-15 16:35:40 UTC
(In reply to Jonathan Druart from comment #46)
> After discussion with Mirko, we have found some others issues with the
> --letsencrypt param:
> 
> check_letsencrypt should not be called unless the param is passed

Patch added.

> If you are a liar and the symlink does not exist, you get a weird error from
> adduser (adduser: Please enter a username matching the regular expression
> configured)

That was actually due to a too short instance name. You get the same if you do it without --letsencrypt.

> sudo koha-create --create-db LE
> adduser: Bitte geben Sie einen Benutzernamen ein, der die Kriterien des
> regulären Ausdrucks erfüllt, welcher in der Konfigurationsvariablen NAME_REGEX > festgelegt ist. Verwenden Sie die Option »--force-badname«, um
> die Überprüfung weniger strikt durchzuführen, oder ändern Sie NAME_REGEX.
Comment 49 Jonathan Druart 2016-04-18 08:32:41 UTC
Created attachment 50307 [details] [review]
Bug 15303 Letsencrypt option for Debian package installations

New option koha-create --letsencrypt

- installs the letsencrypt package if needed
- creates <instance>
- generates letsencrypt certificates for <instance>
- sets up a https-only website for <instance>
- redirects http to https for <instance>

! you need to enable jessie backports to install letsencrypt: add
deb http://http.debian.net/debian jessie-backports main contrib non-free
to your /etc/apt/sources.list

! this patch uses the letsencrypt staging server
to create real certificates, apply thy "LE production server" patch

Test plan:
- build a debian package with patch applied
- use apache mod_ssl
  sudo a2enmod ssl
- make sure the machine is accessible on 80 (needed for letsencrypt) and 443 from the internet
- install koha with your new package
- Put your (existing) domain options in /etc/koha/koha-sites.conf
- use koha-create with the new options:
  sudo koha-create --create-db --letsencrypt <instance>
- if you do not have the letsencrypt package installed, you will be prompted to do that
  [
    if there is no package available, a symlink to the git checkout will work:

    on your test server, get letsencrypt via git
    git clone https://github.com/letsencrypt/letsencrypt

    create a symlink from /usr/bin/letsencrypt to letsencrypt-auto
    sudo ln -s /path/to/letsencrypt/letsencrypt-auto /usr/bin/letsencrypt
  ]

- wait until setup is finished, check that you got a working OPAC and staff client with certificates
- check that http redirects to https

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 50 Jonathan Druart 2016-04-18 08:32:45 UTC
Created attachment 50308 [details] [review]
Bug 15303 LE production server

Deletes the --staging option of the letsencrypt command to get
real certificates. Rate limits apply.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 51 Jonathan Druart 2016-04-18 08:32:51 UTC
Created attachment 50309 [details] [review]
Bug 15303 Followup for koha-remove

This patch makes koha-remove take care of

/etc/letsencrypt/renewal/$opacdomain.conf
/var/lib/koha/$site/letsencrypt.enabled

It also adds a few helper functions.

To test
- create a koha instance with LE
- observe you got both files mentioned above
- remove that instance
- verify the files are gone

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 52 Jonathan Druart 2016-04-18 08:32:58 UTC
Created attachment 50310 [details] [review]
Bug 15303 Followup: Fix apt-cache show breaking koha-create

Temporarely set +e so the test for the LE package does not break
koha-create

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 53 Jonathan Druart 2016-04-18 08:33:03 UTC
Created attachment 50311 [details] [review]
Bug 15303 QA-Followup: check_letsencrypt only if --letsencrypt is used

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 54 Jonathan Druart 2016-04-18 08:33:07 UTC
Created attachment 50312 [details] [review]
Bug 15303: Test if the symlink to letsencrypt-auto exists

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 55 Jonathan Druart 2016-04-18 08:36:25 UTC
(In reply to Mirko Tietgen from comment #48)
> > If you are a liar and the symlink does not exist, you get a weird error from
> > adduser (adduser: Please enter a username matching the regular expression
> > configured)
> 
> That was actually due to a too short instance name. You get the same if you
> do it without --letsencrypt.

No, it's because I used uppercase characters.
Comment 56 Mirko Tietgen 2016-04-18 09:04:22 UTC Comment hidden (obsolete)
Comment 57 Jonathan Druart 2016-04-18 10:19:26 UTC
Comment on attachment 50313 [details] [review]
Bug 15303 Followup: If install LE dependencies from backports on Jessie

Review of attachment 50313 [details] [review]:
-----------------------------------------------------------------

::: debian/scripts/koha-create
@@ +338,4 @@
>          if [ $aptcacheshow -eq 0 ]; then
>                  read -r -p "The letsencrypt package is not installed. Do it now?  [y/N] " response
>                  if [[ $response =~ ^([yY][eE][sS]|[yY])$ ]]; then
> +                    local debrelease="$(lsb_release -c | sed 's|.*\W\(.*\)|\1|')"

lsb-release -c -s
seems to be the way to do :)
Comment 58 Mirko Tietgen 2016-04-18 10:52:04 UTC
Oh, great catch! :D I blindly copied it from something I wrote a while ago. I will make a change.
Comment 59 Mirko Tietgen 2016-04-18 12:55:37 UTC
Created attachment 50324 [details] [review]
Bug 15303 Followup: If install LE dependencies from backports on Jessie

On Debian Jessie, apt needs to be told it should get the dependencies
for the LE package from backports, or it will error. We check if we are
on Jessie and do that automatically so users don't need to set priorities
manually in apt.

Changed to lsb_release -c -s.
Comment 60 Kyle M Hall 2016-04-29 13:06:39 UTC
Pushed to master for the May 2016 release. Gracias Mirko!
Comment 61 David Cook 2016-05-02 00:18:17 UTC
Woo! Hurray Mirko!