Truth is our test suite has been exploding consistently when trying to launch OpenSearch: * https://jenkins.koha-community.org/job/Koha_Main_OS2/ * https://jenkins.koha-community.org/job/Koha_Main_OS1/ This has some precedent discussion on [issue 429](https://gitlab.com/koha-community/koha-testing-docker/-/issues/429) in `KTD` project issues. We are not going to remove it, I guess. But some noise is needed as people using OpenSearch in production don't seem to care we are not being able to run the test suite, so probably promising some 'OpenSearch support' that we cannot guarantee in any way. And if there's a problem with OpenSearch, we (devs) cannot launch a working environment to fix/test. My two cents on this matter.
This is the last Jenkins run before it got borked and never worked again: https://jenkins.koha-community.org/job/Koha_Main_OS2/38
(In reply to Tomás Cohen Arazi (tcohen) from comment #0) > We are not going to remove it, I guess. But some noise is needed as people > using OpenSearch in production don't seem to care we are not being able to > run the test suite, so probably promising some 'OpenSearch support' that we > cannot guarantee in any way. And if there's a problem with OpenSearch, we > (devs) cannot launch a working environment to fix/test. As someone using OpenSearch in production, I wouldn't say that I don't care. I totally forgot that this was an issue, so thanks for making the noise and drawing my attention. I thought that I provided some advice on this one previously, but I can't recall where now. With my koha-testing-docker setup on Windows, I don't think the "ktd" command works, so I don't/can't use it. I'll look at figuring out my own test plan.
Based off a comment on bug 33563 maybe it was just something Magnus and I discussed privately for a prod install...
(In reply to David Cook from comment #2) > With my koha-testing-docker setup on Windows, I don't think the "ktd" > command works, so I don't/can't use it. Yep, had another crack at using the "ktd" script on the Bash in Git for Windows, and no dice. I probably should switch to using WSL or something like that, but not today...
(In reply to David Cook from comment #4) > Yep, had another crack at using the "ktd" script on the Bash in Git for > Windows, and no dice. I probably should switch to using WSL or something > like that, but not today... And the networking is borked on my WSL so that won't work either 🤣
As I note on the koha-testing-docker Gitlab issue, it appears that version 8 of the Search::Elasticsearch Perl library arbitrarily prevents connections to Opensearch (and any other Elasticsearch-based product that doesn't provide a particular HTTP header in its HTTP response). Technically speaking, due to the lack of encapsulation in Perl code, it's trivial to turn off this product checking from Koha code. I suppose the question is whether or not people want to workaround this feature of the Search::Elasticsearch Perl library. As I note on the koha-testing-docker Gitlab issue, we can get the Opensearch integration working again, but do we want to?
This is how you'd do it if you wanted to: diff --git a/Koha/SearchEngine/Elasticsearch.pm b/Koha/SearchEngine/Elasticsearch.pm index 5ef5235720..1cee774d41 100644 --- a/Koha/SearchEngine/Elasticsearch.pm +++ b/Koha/SearchEngine/Elasticsearch.pm @@ -38,6 +38,8 @@ use Clone qw( clone ); use Modern::Perl; use Readonly qw( Readonly ); use Search::Elasticsearch; +use Search::Elasticsearch::Role::Cxn; +$Search::Elasticsearch::Role::Cxn::PRODUCT_CHECK_VALUE = ''; use Try::Tiny qw( catch try ); use YAML::XS;
Keeping in mind that in the future the variable scoping of that variable could change and even the looseness of Perl wouldn't prevent this workaround from failing.
Of course, you could put a proxy in front of your Opensearch and inject the HTTP header. Or you could clone Search::Elasticsearch::Cxn::HTTPTiny, write a hack into a new version, and then update the config in koha-conf.xml to point to that new Cxn library. Lots of possible workarounds, but it would be going against the intention of the library. I'll leave it up to you folk to determine what you'd like to do.
(In reply to David Cook from comment #7) > This is how you'd do it if you wanted to: > > diff --git a/Koha/SearchEngine/Elasticsearch.pm > b/Koha/SearchEngine/Elasticsearch.pm > index 5ef5235720..1cee774d41 100644 > --- a/Koha/SearchEngine/Elasticsearch.pm > +++ b/Koha/SearchEngine/Elasticsearch.pm > @@ -38,6 +38,8 @@ use Clone qw( clone ); > use Modern::Perl; > use Readonly qw( Readonly ); > use Search::Elasticsearch; > +use Search::Elasticsearch::Role::Cxn; > +$Search::Elasticsearch::Role::Cxn::PRODUCT_CHECK_VALUE = ''; > use Try::Tiny qw( catch try ); > use YAML::XS; Let's go with this. That said, I launched an OpenSearch instance on DigitalOcean and didn't manage to make this work.
(In reply to Tomás Cohen Arazi (tcohen) from comment #10) > (In reply to David Cook from comment #7) > > +use Search::Elasticsearch::Role::Cxn; > > +$Search::Elasticsearch::Role::Cxn::PRODUCT_CHECK_VALUE = ''; > > use Try::Tiny qw( catch try ); > > use YAML::XS; > > Let's go with this. That said, I launched an OpenSearch instance on > DigitalOcean and didn't manage to make this work. Were you able to get it working with koha-testing-docker? Sometime in the next year, I'll probably be doing a series of upgrades, which will expose me to the problem. That might not be helpful, but I'd have more real-world info then...
Just looking at my history... and it looks like I used "koha/elasticsearch-icu:opensearch-2.x" for my testing.
If you couldn't get it to work, you might also need to add the following: $Search::Elasticsearch::Role::Cxn::PRODUCT_CHECK_HEADER= 'NONEXISTENT_HEADER'; -- If we patch this like I described before, it would disable the product check for all ES/OS. Is that what we'd want?
(In reply to David Cook from comment #7) > This is how you'd do it if you wanted to: > > diff --git a/Koha/SearchEngine/Elasticsearch.pm > b/Koha/SearchEngine/Elasticsearch.pm > index 5ef5235720..1cee774d41 100644 > --- a/Koha/SearchEngine/Elasticsearch.pm > +++ b/Koha/SearchEngine/Elasticsearch.pm > @@ -38,6 +38,8 @@ use Clone qw( clone ); > use Modern::Perl; > use Readonly qw( Readonly ); > use Search::Elasticsearch; > +use Search::Elasticsearch::Role::Cxn; > +$Search::Elasticsearch::Role::Cxn::PRODUCT_CHECK_VALUE = ''; > use Try::Tiny qw( catch try ); > use YAML::XS; Just applied this patch and had a go, and it seemed to work for me. There are a few failing tests, but it launches and is searchable. I'll see if I can figure out why the test is failing. For reference: > [10:04:49] t/db_dependent/Search.t ........................................ > Dubious, test returned 255 (wstat 65280, 0xff00) > Failed 3/3 subtests
(In reply to Jake Deery from comment #14) > Just applied this patch and had a go, and it seemed to work for me. There > are a few failing tests, but it launches and is searchable. I'll see if I > can figure out why the test is failing. For reference: > > > [10:04:49] t/db_dependent/Search.t ........................................ > > Dubious, test returned 255 (wstat 65280, 0xff00) > > Failed 3/3 subtests I'm taking a look at t/db_dependent/Search.t and I don't think it actually connects to Elasticsearch/OS, so that's interesting. What are the tests that are failing?
any updates here?
(In reply to Jonathan Druart from comment #16) > any updates here? I'm still using OpenSearch in prod, so if anyone has any questions/concerns I'm happy to look at them. Do we want to use the workaround I suggested or leave it up to people to customize Koha and use the workaround there?
I'm happy to add the workaround instructions to the installation docs. If we can fix it (and someone does the patches), I think we should continue to support it. (Particularly for those where licensing is important for their tech stack.)
(In reply to David Nind from comment #18) > I'm happy to add the workaround instructions to the installation docs. > > If we can fix it (and someone does the patches), I think we should continue > to support it. (Particularly for those where licensing is important for > their tech stack.) Well the workaround is to the code so that makes it trickier. They'd have to maintain their own local version to avoid it getting splatted with regular updates.
It also won't fix the 2 failing Jenkins nodes. So I guess the question is if the workaround is something we could ship or not?
(In reply to David Cook from comment #19) > (In reply to David Nind from comment #18) > > I'm happy to add the workaround instructions to the installation docs. > > > > If we can fix it (and someone does the patches), I think we should continue > > to support it. (Particularly for those where licensing is important for > > their tech stack.) > > Well the workaround is to the code so that makes it trickier. They'd have to > maintain their own local version to avoid it getting splatted with regular > updates. I'd say: go ahead and propose to ship our patched version of the library. Talk to Mason about dependencies tweaks in the debian control files. This is all non sense IMHO. They went roge and disabled the competition usage. Just ship a fixed version.
> I'd say: go ahead and propose to ship our patched version of the library. I though the only proposed patch here and on the KTD ticket was on Koha/SearchEngine/Elasticsearch.pm ? And not on Search::Elasticsearch itself.
Even if forking Search::Elasticsearch wasn't really yet on the table for OpenSearch support, it might be needed sooner than we think for all future Elasticsearch support. See Bug 39174: Search::Elasticsearch is being retired. And if ES 9 (coming likely in summer or September) doesn't work out of the box then that will be it. No support. And if we don't find a way to make ES 9 work before 26.11 LTS release, it will end up stuck with only supporting ES 8 which will very likely be EOL before 2030-05 ------- There is an OpenSearch perl client: https://metacpan.org/dist/OpenSearch Looking at the multiple release changelogs, I guess it should do the job for base features, but it certainly doesn't support Elasticsearch (it's not a fork of Search::Elasticsearch) But there still might be a path there in helping to add ES support. Since there is no ES perl client anymore, if we are lucky, it might be welcomed there to have contributions from people using ES with Perl. And we are not the only ones interested in the future of ES Perl support: https://github.com/elastic/elasticsearch-perl/issues/220 So that's another option. In addition with forking Search::Elasticsearch and trying to add future OpenSearch 3.X support. And maybe proposing to the people making the OpenSearch perl lib to join forces here. Devs at ES seem open to giving some pointers to help supporting future ES versions with perl: https://github.com/elastic/elasticsearch-perl/issues/220#issuecomment-1196582337 Which would still help even if going the path of adding ES support to the OpenSearch perl lib. Also at Opensearch there might be some support: https://opensearch.org/blog/community-clients/ It lists elasticsearch-perl as needing a fork. Which would get help. Any of those paths might benefit from a name not tied to one of the products to make more clear to the users that the support is dual. And to have more chances getting some help when need from the two products. Since the user base is small, it might be tough enough already. (Just seeding the idea for later.)
(In reply to Tomás Cohen Arazi (tcohen) from comment #21) > This is all non sense IMHO. They went rogue and disabled the competition > usage. Even though ES went back to using a libre/open source license. There was lost confidence in the whole journey. I didn't know during that time they also did this. That helps even less to regain confidence... So one more reason part of people might feel that AWS's OpenSearch it still a safer option. And that if reasonably possible, there is still value in having it working in Koha :/ (news about OS: even if it's now under the umbrella of the Linux Foundation, still ~90% of top 30 contributors last year are AWS employees) Both parties made somewhat logical decisions/bets to maximize their profits before anything else and in the end we pay a price in totally useless work 🤬. Especially in a language without first class library support. (having to support two libs would have been quite the useless work also) > They went rogue and disabled the competition usage. I quickly looked at the main official OpenSearch clients and it seems they don't even support Elasticsearch so everyone everywhere using ES/OS is screwed in some way for dual support. Back to the above paragraph...
@Victor: That's good to know! Worst case scenario: We write our own client. The handy thing about OS/ES is that it's a HTTP API. Doing searches against the HTTP API is super easy. If we struggled with the indexing side of things, we could always implement the background worker in another language with better support. But it sucks that we're in this position :/.