Bug 33563

Summary: Document Elasticsearch secure mode
Product: Koha Reporter: Liz Rea <wizzyrea>
Component: SearchingAssignee: David Cook <dcook>
Status: Pushed to oldstable --- QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: andreas.jonsson, dcook, fridolin.somers, lucas, magnus, wainuiwitikapark, wizzyrea, wsmith
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
When using authentication with Elasticsearch/Opensearch, you must use HTTPS. This change adds some comments in koha-conf.xml to show how to do configure Koha to use authentication and HTTPS for ES/OS.
 Version(s) released in:
24.11.00,24.05.02,23.11.07
Circulation function:
Attachments: Bug 33563: Add comments for Elasticsearch security config
Bug 33563: Add comments for Elasticsearch security config
Bug 33563: Add comments for Elasticsearch security config

Description Liz Rea 2023-04-19 16:27:04 UTC 
Hi friends,
Koha should support authenticated access to Elasticsearch, at the very minimum https://www.elastic.co/guide/en/elasticsearch/reference/current/http-clients.html and https://www.elastic.co/guide/en/elasticsearch/reference/current/token-authentication-services.html

We'd need a place to stash our ES configuration information much like we used to for Zebra. koha-conf.xml is probably fine?

Obviously open to discussion from the way smarter people here - this is only to get the discussion started, as Elasticsearch will start creating egregious scary logs for every connection complaining about security, starting with version 7. 

Liz
Comment 1 Andreas Jonsson 2023-05-03 17:43:45 UTC 
For basic authentication, you can put the credentials in the connection details:

  <server>username:password@localhost:9200</server>
   
It would be nice to have a comment about this in the koha-conf.xml template.
Comment 2 Liz Rea 2023-05-08 17:07:23 UTC 
Agreed, let's make the bug "document secure mode" :D
Comment 3 David Cook 2024-02-02 01:16:33 UTC 
I've been setting up Elasticsearch 7 with a Koha and this is what I used:

 <!-- Elasticsearch Configuration -->
 <elasticsearch>
     <server>localhost:9200</server> <!-- may be repeated to include all servers on your cluster -->
     <userinfo>elastic:CHANGEME</userinfo>
     <use_https>1</use_https>

The relevant bits above being "userinfo" and "use_https", and I found them by looking at https://metacpan.org/pod/Search::Elasticsearch::Role::Cxn#node and looking around the code a bit.

With Elasticsearch 7, to use basic auth you have to use HTTPS as well, but that's a much more complicated topic. We could add commented out "userinfo" and "use_https" elements in koha-conf.xml though to get people on the right track.
Comment 4 David Cook 2024-05-22 23:51:21 UTC Comment hidden (obsolete)
Comment 5 David Cook 2024-05-22 23:56:15 UTC 
I helped out Magnus on Mattermost with setting this up for Opensearch, so I thought it was timely to just write a patch and get it in. 

Note that the comment asserts that if you're using authentication then you will need to use HTTPS. That's really the key part. 

--

Opensearch comes with self-signed certs out of the box, so it's very easy to use HTTPS with it. 

With Elasticsearch, you have to set up your own self-signed certs, but I think that's outside the scope of this bug. You can consult Elasticsearch's documentation for setting up the server-side for HTTPS.
Comment 6 David Cook 2024-05-23 05:56:51 UTC 
Now I'm second guessing my comment...

I thought that you had to use HTTPS if you were using authentication with ES...

But in koha-testing-docker we don't have HTTPS enabled for Elasticsearch, but if you add userinfo and no use_https "koha-elasticsearch -v -b --rebuild kohadev" will work...
Comment 7 Magnus Enger 2024-05-23 07:02:48 UTC 
Created attachment 167066 [details] [review]
Bug 33563: Add comments for Elasticsearch security config

This patch adds some commented Elasticsearch security configuration,
which shows how to use username/password with HTTPS.

Test plan:
0. Apply patch
1. cp debian/templates/koha-conf-site.xml.in /etc/koha/koha-conf-site.xml.in
2. koha-create --create-db test
3. vi /etc/koha/sites/test/koha-conf.xml
4. Note that the comments for userinfo and use_https are in the koha-conf.xml

Signed-off-by: Magnus Enger <magnus@libriotech.no>
Works as advertised.
Comment 8 Magnus Enger 2024-05-23 07:06:37 UTC 
The patch does what it says on the tin, but I have not looked into the question of auth with or without https.
Comment 9 David Cook 2024-05-23 07:08:24 UTC 
(In reply to Magnus Enger from comment #8)
> The patch does what it says on the tin, but I have not looked into the
> question of auth with or without https.

I suppose at the very least it's handy to have the commented out fields there
Comment 10 Nick Clemens (kidclamp) 2024-06-25 17:49:30 UTC 
Created attachment 168093 [details] [review]
Bug 33563: Add comments for Elasticsearch security config

This patch adds some commented Elasticsearch security configuration,
which shows how to use username/password with HTTPS.

Test plan:
0. Apply patch
1. cp debian/templates/koha-conf-site.xml.in /etc/koha/koha-conf-site.xml.in
2. koha-create --create-db test
3. vi /etc/koha/sites/test/koha-conf.xml
4. Note that the comments for userinfo and use_https are in the koha-conf.xml

Signed-off-by: Magnus Enger <magnus@libriotech.no>
Works as advertised.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 11 Katrin Fischer 2024-06-27 06:49:43 UTC 
Always love documentation :)
Comment 12 Katrin Fischer 2024-06-27 09:52:19 UTC 
Pushed for 24.11!

Well done everyone, thank you!
Comment 13 Lucas Gass (lukeg) 2024-07-18 19:59:06 UTC 
Backported to 24.05.x for upcoming 24.05.02
Comment 14 Fridolin Somers 2024-07-19 09:28:24 UTC 
Pushed to 23.11.x for 23.11.07
Comment 15 Wainui Witika-Park 2024-09-12 07:10:04 UTC 
Not backporting to 23.05.x unless requested