Bug 26893

Summary: New version of JSON::Validator (D11) breaks our REST API routes
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: Architecture, internals, and plumbingAssignee: Mason James <mtj>
Status: RESOLVED DUPLICATE QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: b.pastern4k, dcook, emmi.takkinen, ere.maijala, julian.maurice, martin.renvoize, mtj, nick, tomascohen
Version: unspecified   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19735
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17092
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28800
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29964
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30193
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 26893: Limit JSON::Validator versions

Description Jonathan Druart 2020-11-02 11:16:18 UTC
Error while loading /etc/koha/sites/kohadev/plack.psgi: Can't load application from file "/kohadevbox/koha/api/v1/app.pl": Can't use string ("paths.json") as a HASH ref while "strict refs" in use at /usr/share/perl5/JSON/Validator/OpenAPI/Mojolicious.pm line 333.
Comment 1 Jonathan Druart 2020-11-02 11:20:04 UTC
The tests were passing on Oct 22th (run 115) but fail since Oct 26th (run 123)

https://jenkins.koha-community.org/job/Koha_Master_D11/
Comment 2 Julian Maurice 2020-11-02 14:56:27 UTC
Confirmed. The problem exists for all versions of JSON::Validator from 4.06 to 4.10
Comment 3 Martin Renvoize 2020-11-06 15:37:20 UTC
Created attachment 113254 [details] [review]
Bug 26893: Limit JSON::Validator versions
Comment 4 Martin Renvoize 2020-11-06 15:38:09 UTC
We need to varify whether the package build process takes notice of the cpanfile min, max and excludes versions.. I can't remember if we got that far, though I do know mason was working on it.
Comment 5 Martin Renvoize 2020-11-06 15:39:01 UTC
Longer term it would be nice to get the latest Mojolicious::Plugin::OpenAPI building on D11.. but at the moment that seems to fail.
Comment 6 Jonathan Druart 2020-11-06 15:54:38 UTC
Looks like we cannot force the version:

% apt install libjson-validator-perl=3.25+dfsg-1+koha3

Error while loading /etc/koha/sites/kohadev/plack.psgi: Can't load application from file "/kohadevbox/koha/api/v1/app.pl": Can't locate JSON/Validator/Util.pm in @INC (you may need to install the JSON::Validator::Util module) (@INC contains: /kohadevbox/koha /kohadevbox/koha/installer /kohadevbox/koha/lib/installer /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.3 /usr/local/share/perl/5.30.3 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl-base /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /var/lib/koha/kohadev/plugins) at /usr/local/share/perl/5.30.3/JSON/Validator/OpenAPI/Mojolicious.pm line 5.

Of course new versions of Mojolicious need the new JSON::Validator
Comment 7 Mason James 2020-11-11 22:00:06 UTC
i think the errors that we are encountering is probably caused by J::V 4.00 depreciating some functions and methods

https://metacpan.org/changes/distribution/JSON-Validator

4.00 2020-06-08T10:41:55+0900
 - JSON::Validator::schema() now holds a JSON::Validator::Schema object
   instead of Mojo::JSON::Pointer
 - Add schema classes for Draft4, Draft6 and Draft7
 - Add "duration" and "uuid" formats #210
 - Fix coercing boolean "false" #215
 - Fix not matching "null" should also be a "type" error #217
 - Deprecated JSON::Validator::joi()
 - Deprecated JSON::Validator::singleton()
 - Deprecated JSON::Validator::validate_json()
 - Deprecated JSON::Validator::version()
 - Removed JSON::Validator::generate_definitions_path()
 - Removed support for JSON::Validator::bundle({ref_key => ...})


i think the best solution would be to attempt to rewrite the JV code in koha to not use the depreciated JV functions and methods

we will eventually have to correct this problem, so lets do it now rather than later?
Comment 8 David Cook 2020-11-11 22:07:25 UTC
(In reply to Mason James from comment #7)
> i think the best solution would be to attempt to rewrite the JV code in koha
> to not use the depreciated JV functions and methods
> 
> we will eventually have to correct this problem, so lets do it now rather
> than later?

Sounds reasonable if it's possible to do it
Comment 9 Martin Renvoize 2020-11-12 14:42:12 UTC
In my understanding we aren't using any of those methods directly with Koha... it's rather than many of those methods are used in Mojolicious::Plugin::OpenAPI which we're using.  Said plugin has been updated to reflect the changes in JSON::Validator but it also requires a new Mojolicious than we're currently supporting I think..

Hense, I don't think it's just a case of updating Koha code, but rather tracking down the versions we wish to support and pinning ourselves to them properly again.
Comment 10 David Cook 2020-11-12 23:30:10 UTC
(In reply to Martin Renvoize from comment #9)
> In my understanding we aren't using any of those methods directly with
> Koha... it's rather than many of those methods are used in
> Mojolicious::Plugin::OpenAPI which we're using.  Said plugin has been
> updated to reflect the changes in JSON::Validator but it also requires a new
> Mojolicious than we're currently supporting I think..
> 
> Hense, I don't think it's just a case of updating Koha code, but rather
> tracking down the versions we wish to support and pinning ourselves to them
> properly again.

I had a feeling that might be the case. 

Alternatively, could we pin JSON::Validator to an older version? 

I suppose every approach has its trade offs...
Comment 11 Jonathan Druart 2020-11-13 07:52:44 UTC
(In reply to David Cook from comment #10)
> Alternatively, could we pin JSON::Validator to an older version? 

See comment 6, that does not work.
Comment 12 David Cook 2020-11-15 23:18:24 UTC
(In reply to Jonathan Druart from comment #11)
> (In reply to David Cook from comment #10)
> > Alternatively, could we pin JSON::Validator to an older version? 
> 
> See comment 6, that does not work.

After doing a bit of digging, comment 6 appears to likely be mistaken. I'll include my detailed comments below.

(In reply to Jonathan Druart from comment #6)
> Looks like we cannot force the version:
> 
> % apt install libjson-validator-perl=3.25+dfsg-1+koha3
> 
> Error while loading /etc/koha/sites/kohadev/plack.psgi: Can't load
> application from file "/kohadevbox/koha/api/v1/app.pl": Can't locate
> JSON/Validator/Util.pm in @INC (you may need to install the
> JSON::Validator::Util module) (@INC contains: /kohadevbox/koha
> /kohadevbox/koha/installer /kohadevbox/koha/lib/installer /etc/perl
> /usr/local/lib/x86_64-linux-gnu/perl/5.30.3 /usr/local/share/perl/5.30.3
> /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5
> /usr/lib/x86_64-linux-gnu/perl-base /usr/lib/x86_64-linux-gnu/perl/5.30
> /usr/share/perl/5.30 /usr/local/lib/site_perl /var/lib/koha/kohadev/plugins)
> at /usr/local/share/perl/5.30.3/JSON/Validator/OpenAPI/Mojolicious.pm line 5.
> 

According to https://metacpan.org/release/JHTHORSEN/JSON-Validator-3.25, JSON::Validator::Util should have been installed. 

I've downloaded the package from http://debian.koha-community.org/koha-staging/pool/main/libj/libjson-validator-perl/libjson-validator-perl_3.25+dfsg-1+koha3_all.deb. Using 7-zip, I've opened the files and I see that Validator.pm is *not* actually version 3.25 but *is* actually version 3.14. JSON::Validator::Util is not included until JSON::Validator version 3.20 it seems.

I'm looking at "apt-cache show libmojolicious-plugin-openapi-perl" on debian:bullseye, and it indicates this dependency: libjson-validator-perl (>= 3.15).

So it's not surprising that libjson-validator-perl=3.25+dfsg-1+koha3 didn't work since it's actually been mis-packaged it seems. 

I think that if we re-package libjson-validator-perl to actually be 3.15+ then it'll probably work. 

In your example, it's interesting that JSON::Validator::OpenAPI::Mojolicious is at /usr/local/share/perl/5.30.3/JSON/Validator/OpenAPI/Mojolicious.pm instead of /usr/share/perl5/JSON/Validator/OpenAPI/Mojolicious.pm. Did you install it using CPAN instead of DEB package?
Comment 13 Jonathan Druart 2020-11-18 14:38:00 UTC
Good catch David!

So, on a fresh D11:

% cpanm https://cpan.metacpan.org/authors/id/J/JH/JHTHORSEN/JSON-Validator-3.25.tar.gz

And that's it, it works!
Comment 14 Jonathan Druart 2020-11-18 14:38:55 UTC
(In reply to David Cook from comment #12)
> In your example, it's interesting that JSON::Validator::OpenAPI::Mojolicious
> is at /usr/local/share/perl/5.30.3/JSON/Validator/OpenAPI/Mojolicious.pm
> instead of /usr/share/perl5/JSON/Validator/OpenAPI/Mojolicious.pm. Did you
> install it using CPAN instead of DEB package?

Yes, I was certainly trying something and installed it from CPAN.
Comment 15 Mason James 2020-11-26 00:54:58 UTC
i've built a new J::V v3.25 package and added it to the 'bullseye' component. which has fixed the deb11 builds

the package is named 'libjson-validator-perl-4.10+really3.25-koha1', following the naming syntax suggested in section 5.6.12.1 below...
 https://www.debian.org/doc/debian-policy/ch-controlfields.html


installing koha on deb11 now currently requires 'bullseye' to be included in the apt repo definition,  example below...

 $ echo 'deb http://debian.koha-community.org/koha stable main bullseye' | sudo tee /etc/apt/sources.list.d/koha.list
Comment 16 Mason James 2020-11-26 01:18:26 UTC
(In reply to David Cook from comment #10)
> (In reply to Martin Renvoize from comment #9)
> > In my understanding we aren't using any of those methods directly with
> > Koha... it's rather than many of those methods are used in
> > Mojolicious::Plugin::OpenAPI which we're using.  Said plugin has been
> > updated to reflect the changes in JSON::Validator but it also requires a new
> > Mojolicious than we're currently supporting I think..
> > 
> > Hense, I don't think it's just a case of updating Koha code, but rather
> > tracking down the versions we wish to support and pinning ourselves to them
> > properly again.
> 
> I had a feeling that might be the case. 
> 
> Alternatively, could we pin JSON::Validator to an older version? 
> 

i asked for 'pinning' advice in the debian #packaging irc channel

it's impossible to pin an older package without instructing users to manually add an entry to their /etc/apt/preferences.d dir

we could automate that task within koha-common, or another helper package - but that is probably considered inappropriate behaviour for a package (for security reasons)


the workaround for pinning, is to name the package using the +really convention described here ->  https://www.debian.org/doc/debian-policy/ch-controlfields.html
Comment 17 David Cook 2020-11-26 03:49:00 UTC
(In reply to Mason James from comment #16)
>
> i asked for 'pinning' advice in the debian #packaging irc channel
> 
> it's impossible to pin an older package without instructing users to
> manually add an entry to their /etc/apt/preferences.d dir
>

Ahh right. In hindsight, I have always manually added entries for pinning. Silly me. 
 
> we could automate that task within koha-common, or another helper package -
> but that is probably considered inappropriate behaviour for a package (for
> security reasons)
> 
> 
> the workaround for pinning, is to name the package using the +really
> convention described here -> 
> https://www.debian.org/doc/debian-policy/ch-controlfields.html

Interesting! I'd never heard of +really, although that still sounds like a bit of a different scenario.

I think that there is a 3rd option. With HTTP::OAI, we use the older version in Koha's Apt repo and don't use the Debian version which is too new. 

"libhttp-oai-perl (>= 3.2) | libhttp-oai-3.27-perl, libhttp-oai-perl (<< 4.0) | libhttp-oai-3.27-perl"

http://debian.koha-community.org/koha/pool/main/libh/
https://packages.debian.org/bullseye/libhttp-oai-perl

That seems like the same thing that we'd want to do here.
Comment 18 Mason James 2020-11-27 01:09:05 UTC
> I think that there is a 3rd option. With HTTP::OAI, we use the older version
> in Koha's Apt repo and don't use the Debian version which is too new. 
> 
> "libhttp-oai-perl (>= 3.2) | libhttp-oai-3.27-perl, libhttp-oai-perl (<<
> 4.0) | libhttp-oai-3.27-perl"
> 
> http://debian.koha-community.org/koha/pool/main/libh/
> https://packages.debian.org/bullseye/libhttp-oai-perl
> 
> That seems like the same thing that we'd want to do here.

i think the 3rd option won't work here, as bulleyes' libmojolicious-plugin-openapi-perl (3.40-1) depends on libjson-validator-perl (>= 4.05)

the libhttp-oai-3.27-perl package works because only the koha pkgs depend on it
Comment 19 Mason James 2020-11-27 01:52:16 UTC
(In reply to Mason James from comment #18)
> > I think that there is a 3rd option. With HTTP::OAI, we use the older version
> > in Koha's Apt repo and don't use the Debian version which is too new. 
> > 
> > "libhttp-oai-perl (>= 3.2) | libhttp-oai-3.27-perl, libhttp-oai-perl (<<
> > 4.0) | libhttp-oai-3.27-perl"
> > 
> > http://debian.koha-community.org/koha/pool/main/libh/
> > https://packages.debian.org/bullseye/libhttp-oai-perl
> > 
> > That seems like the same thing that we'd want to do here.
> 
> i think the 3rd option won't work here, as bulleyes'
> libmojolicious-plugin-openapi-perl (3.40-1) depends on
> libjson-validator-perl (>= 4.05)
> 
> the libhttp-oai-3.27-perl package works because only the koha pkgs depend on
> it

hmm, think about this a bit more... this could work if we built a similarly named  pkg for libmojolicious-plugin-openapi-perl too

this solution might also fix some other issues around support for these 'problematic' packages and older debian versions
Comment 20 David Cook 2020-11-27 03:52:14 UTC
(In reply to Mason James from comment #18)
> i think the 3rd option won't work here, as bulleyes'
> libmojolicious-plugin-openapi-perl (3.40-1) depends on
> libjson-validator-perl (>= 4.05)
> 
> the libhttp-oai-3.27-perl package works because only the koha pkgs depend on
> it

That's interesting... because on November 15th I installed libmojolicious-plugin-openapi-perl on debian:bullseye in Docker and it had this dependency: libjson-validator-perl (>= 3.15).

Yet https://packages.debian.org/bullseye/libmojolicious-plugin-openapi-perl clearly says "dep: libjson-validator-perl (>= 4.05)"

That's weird. I must have had an older version of libmojolicious-plugin-openapi-perl installed somehow...

I suppose the solution is to require older versions of both libmojolicious-plugin-openapi-perl and libjson-validator-perl in that case.
Comment 21 David Cook 2020-11-27 03:57:39 UTC
(In reply to David Cook from comment #20)
> That's interesting... because on November 15th I installed
> libmojolicious-plugin-openapi-perl on debian:bullseye in Docker and it had
> this dependency: libjson-validator-perl (>= 3.15).
> 
> Yet https://packages.debian.org/bullseye/libmojolicious-plugin-openapi-perl
> clearly says "dep: libjson-validator-perl (>= 4.05)"
> 
> That's weird. I must have had an older version of
> libmojolicious-plugin-openapi-perl installed somehow...
> 
> I suppose the solution is to require older versions of both
> libmojolicious-plugin-openapi-perl and libjson-validator-perl in that case.

Ahhhh... it's called "unstable" for a reason.

https://metadata.ftp-master.debian.org/changelogs//main/libm/libmojolicious-plugin-openapi-perl/libmojolicious-plugin-openapi-perl_3.40-1_changelog

On January 29 2020 they list libmojolicious-plugin-openapi-perl (2.21-1).

On November 15 2020 they list libmojolicious-plugin-openapi-perl (3.40-1).

Talk about crazy timing! They uploaded a new package the same day that I was trying out libjson-validator-perl. The change probably happened within hours/days of me checking...

In any case, I'd say yeah let's require a 2.x libmojolicious-plugin-openapi-perl and a 3.x libjson-validator-perl.
Comment 22 Jonathan Druart 2021-01-14 14:24:37 UTC
Lowering severity as it's no longer blocker.

What's next here?
Comment 23 David Cook 2021-01-14 23:49:31 UTC
(In reply to Jonathan Druart from comment #22)
> Lowering severity as it's no longer blocker.
> 
> What's next here?

Good question. It's been a while since I looked at this. 

I think we just need to update debian/control to agree with the cpanfile here.
Comment 24 Jonathan Druart 2021-02-03 14:20:01 UTC
This problem is back!

REST API is broken under D11

root@kohadevbox:~$ apt show libmojolicious-plugin-openapi-perl
Package: libmojolicious-plugin-openapi-perl
Version: 3.40-1
Priority: optional
Section: perl
Maintainer: Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org>
Installed-Size: 195 kB
Depends: perl:any, libdata-validate-domain-perl, libdata-validate-ip-perl, libjson-validator-perl (>= 4.05), libmojolicious-perl (>= 8.00), libtext-markdown-perl (>= 1.000031)
Homepage: https://metacpan.org/release/Mojolicious-Plugin-OpenAPI
Download-Size: 73.8 kB
APT-Manual-Installed: no
APT-Sources: http://httpredir.debian.org/debian bullseye/main amd64 Packages
Description: OpenAPI/Swagger plugin for Mojolicious

N: There is 1 additional record. Please use the '-a' switch to see it
root@kohadevbox:~$ apt show libjson-validator-perl
Package: libjson-validator-perl
Version: 4.12+dfsg-1
Priority: optional
Section: perl
Maintainer: Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org>
Installed-Size: 234 kB
Depends: perl:any, libmojolicious-perl (>= 7.28), libyaml-pp-perl, openapi-specification
Recommends: libcpanel-json-xs-perl (>= 3.0200), libdata-validate-domain-perl, libdata-validate-ip-perl, libnet-idn-encode-perl, libscalar-list-utils-perl, libsereal-encoder-perl (>= 4.00)
Breaks: sreview-web (<< 0.6.1-2)
Replaces: sreview-web (<< 0.6.1-2)
Homepage: https://metacpan.org/release/JSON-Validator
Tag: devel::lang:perl, devel::library, implemented-in::perl
Download-Size: 80.8 kB
APT-Manual-Installed: no
APT-Sources: http://httpredir.debian.org/debian bullseye/main amd64 Packages
Description: module to validate data against a JSON schema

N: There are 2 additional records. Please use the '-a' switch to see them.
root@kohadevbox:~$ apt-cache policy libmojolicious-plugin-openapi-perl
libmojolicious-plugin-openapi-perl:
  Installed: 3.40-1
  Candidate: 3.40-1
  Version table:
 *** 3.40-1 500
        500 http://httpredir.debian.org/debian bullseye/main amd64 Packages
        100 /var/lib/dpkg/status
     2.16-1+koha1 500
        500 http://debian.koha-community.org/koha-staging dev/main amd64 Packages
root@kohadevbox:~$ apt-cache policy libjson-validator-perl
libjson-validator-perl:
  Installed: 4.12+dfsg-1
  Candidate: 4.12+dfsg-1
  Version table:
 *** 4.12+dfsg-1 500
        500 http://httpredir.debian.org/debian bullseye/main amd64 Packages
        100 /var/lib/dpkg/status
     4.10+really3.25-koha1 500
        500 http://debian.koha-community.org/koha-staging dev/bullseye amd64 Packages
     3.25+dfsg-1+koha3 500
        500 http://debian.koha-community.org/koha-staging dev/main amd64 Packages
Comment 25 David Cook 2021-02-04 04:06:34 UTC
(In reply to Jonathan Druart from comment #24)
> This problem is back!
> 
> REST API is broken under D11
> 

If possible, let's require older versions of both libmojolicious-plugin-openapi-perl and libjson-validator-perl. 

I suppose it'll depend on what version of Mojolicious is in Bullseye, although I feel like Mojolicious tends to be backwards compatible, so you should be able to use older plugins on newer frameworks... but I'm not looking at this today.
Comment 26 Martin Renvoize 2021-03-01 16:28:39 UTC
Mojolicious uses pretty standard semantic versioning.. jump a major and you can't really expect things to work.

The latest combination of dependencies I found to work with current Koha is:

Mojolicous 8.73
JSON::Validator 4.05
Mojolicious::Plugin::OpenAPI 3.39

If we wanted to jump beyond that we'll need to change the Koha codebase to accommodate the changes in the libraries.

What's easier, backporting new libraries to older Debian (and updating our codebase for the newer libraries) or downgrading/pinning older libraries on newer Debian?


Or, dare I say it.. shipping a local clone of the Mojo stack inside Koha.
Comment 27 Jonathan Druart 2021-03-01 16:46:36 UTC
The short term solution is to package a 4.99+really3.25-koha
Comment 28 David Cook 2021-03-02 03:25:24 UTC
(In reply to Jonathan Druart from comment #27)
> The short term solution is to package a 4.99+really3.25-koha

Why? 

We should be able to provide the right dependency in the Koha apt repo and specify the right dependency in the control file.
Comment 29 David Cook 2021-03-02 03:28:49 UTC
(In reply to Martin Renvoize from comment #26)
> 
> What's easier, backporting new libraries to older Debian (and updating our
> codebase for the newer libraries) or downgrading/pinning older libraries on
> newer Debian?
> 

I would think probably the latter.

> 
> Or, dare I say it.. shipping a local clone of the Mojo stack inside Koha.

I'm not opposed to this. I've started using Carton on other projects, and it's nice having all my Perl dependencies stored in the "local" directory, and then just pointing to that in PERL5LIB.

That said, without using the cpanfile.snapshot via Carton, this could get very unmanageable and create many headaches.
Comment 30 Mason James 2021-03-03 22:37:08 UTC
(In reply to Jonathan Druart from comment #27)
> The short term solution is to package a 4.99+really3.25-koha

hi Jonathan, this is done

$ cat /etc/apt/sources.list.d/koha.list
deb http://debian.koha-community.org/koha-staging  dev main bullseye

$ apt-cache policy libjson-validator-perl
libjson-validator-perl:
  Installed: 4.99+really3.25-koha1
  Candidate: 4.99+really3.25-koha1
  Version table:
 *** 4.99+really3.25-koha1 500
        500 http://debian.koha-community.org/koha-staging dev/bullseye amd64 Packages
        100 /var/lib/dpkg/status
     4.13+dfsg-1 500
        500 http://apt.kohaaloha.com:3142/ftp.nz.debian.org/debian testing/main amd64 Packages
     3.25+dfsg-1+koha3 500
        500 http://debian.koha-community.org/koha-staging dev/main amd64 Packages
Comment 31 Mason James 2021-03-03 22:49:38 UTC
(In reply to David Cook from comment #28)
> (In reply to Jonathan Druart from comment #27)
> > The short term solution is to package a 4.99+really3.25-koha
> 
> Why? 
> 
> We should be able to provide the right dependency in the Koha apt repo and
> specify the right dependency in the control file.

no, it's impossible for us to do this because the version we want (3.25) is *lower* than the version available (4.13) in the debian.net/bullseye repo

see... https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26893#c16

ps: i would love to be proven incorrect on this info
Comment 32 Tomás Cohen Arazi 2021-03-03 23:26:23 UTC
(In reply to Martin Renvoize from comment #26)
> Mojolicious uses pretty standard semantic versioning.. jump a major and you
> can't really expect things to work.
> 
> The latest combination of dependencies I found to work with current Koha is:
> 
> Mojolicous 8.73
> JSON::Validator 4.05
> Mojolicious::Plugin::OpenAPI 3.39
> 
> If we wanted to jump beyond that we'll need to change the Koha codebase to
> accommodate the changes in the libraries.
> 
> What's easier, backporting new libraries to older Debian (and updating our
> codebase for the newer libraries) or downgrading/pinning older libraries on
> newer Debian?
> 
> 
> Or, dare I say it.. shipping a local clone of the Mojo stack inside Koha.

We should do that. And jump to current versions of the OpenAPI plugin and Mojolicious 8.x. if not possible to package them, of course. I think there was a Perl version issue with stretch?
Comment 33 David Cook 2021-03-05 03:52:06 UTC
(In reply to Mason James from comment #31)
> no, it's impossible for us to do this because the version we want (3.25) is
> *lower* than the version available (4.13) in the debian.net/bullseye repo
> 
> see... https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26893#c16
> 
> ps: i would love to be proven incorrect on this info

Except that we already do it with libhttp-oai-perl. 

At https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26893#c18 you mention the issue becomes libmojolicious-plugin-openapi-perl has dependency issues, but then we can just control that one too and provide our own package, which makes sense since we're already mucking with Mojolicious versions.
Comment 34 David Cook 2021-03-05 03:54:18 UTC
(In reply to Tomás Cohen Arazi from comment #32)
> (In reply to Martin Renvoize from comment #26)
> > Or, dare I say it.. shipping a local clone of the Mojo stack inside Koha.
> 
> We should do that. And jump to current versions of the OpenAPI plugin and
> Mojolicious 8.x. if not possible to package them, of course. I think there
> was a Perl version issue with stretch?

I don't know about stretch, but yeah I suppose the downside of picking and choosing some libraries to be via DEB packages and some to be embedded is that our support for multiple different OSes could be problematic (based on Mojo dependencie).
Comment 35 Jonathan Druart 2021-03-09 10:10:40 UTC
Lowering severity again.
Comment 36 Bohdan 2021-04-20 10:19:58 UTC
Hello. I've installed Koha 20.11.04 on latest Debian 10 and the API is broken. I cannot get the list of available branches as it uses API.

Got the following in api error logs:
[ERROR] Warning: Could not load REST API spec bundle: Can't use string ("parameters.json") as a HASH ref while "strict refs" in use at /usr/share/perl5/JSON/Validator/OpenAPI/Mojolicious.pm line 272.
Comment 37 Mason James 2021-04-20 12:54:46 UTC
(In reply to Bohdan from comment #36)
> Hello. I've installed Koha 20.11.04 on latest Debian 10 and the API is
> broken. I cannot get the list of available branches as it uses API.
> 
> Got the following in api error logs:
> [ERROR] Warning: Could not load REST API spec bundle: Can't use string
> ("parameters.json") as a HASH ref while "strict refs" in use at
> /usr/share/perl5/JSON/Validator/OpenAPI/Mojolicious.pm line 272.

hi Bohdan, 
are you using 64-bit debian? it is recommended
Comment 38 Bohdan 2021-04-20 16:16:05 UTC
(In reply to Mason James from comment #37)
> (In reply to Bohdan from comment #36)
> > Hello. I've installed Koha 20.11.04 on latest Debian 10 and the API is
> > broken. I cannot get the list of available branches as it uses API.
> > 
> > Got the following in api error logs:
> > [ERROR] Warning: Could not load REST API spec bundle: Can't use string
> > ("parameters.json") as a HASH ref while "strict refs" in use at
> > /usr/share/perl5/JSON/Validator/OpenAPI/Mojolicious.pm line 272.
> 
> hi Bohdan, 
> are you using 64-bit debian? it is recommended

Of course, it is x64.
Comment 39 David Cook 2021-04-21 05:01:33 UTC
(In reply to Martin Renvoize from comment #26)
> Mojolicious uses pretty standard semantic versioning.. jump a major and you
> can't really expect things to work.
> 
> The latest combination of dependencies I found to work with current Koha is:
> 
> Mojolicous 8.73
> JSON::Validator 4.05
> Mojolicious::Plugin::OpenAPI 3.39
> 
> If we wanted to jump beyond that we'll need to change the Koha codebase to
> accommodate the changes in the libraries.
> 
> What's easier, backporting new libraries to older Debian (and updating our
> codebase for the newer libraries) or downgrading/pinning older libraries on
> newer Debian?
> 
> 
> Or, dare I say it.. shipping a local clone of the Mojo stack inside Koha.

I'm still investigating, but I'd be interested in getting at least JSON::Validator 4.10. At a glance, it looks like it will work with Mojolicious 7.28. 

Wow... Mojolicious::Plugin::OpenAPI and JSON::Validator have very different ideas about Mojolicious dependencies. JSON::Validator 4.x is OK with Mojolicious 7, but Mojolicious::Plugin::OpenAPI 2.x+ wants Mojolicious 8 and lower JSON::Validator versions...
Comment 40 David Cook 2021-04-21 05:22:19 UTC
Only realising now that Mojolicious::Plugin::OpenAPI is only in Bullseye: https://packages.debian.org/bullseye/libmojolicious-plugin-openapi-perl

It's version 3.41 and requires JSON::Validator 4.11+ and Mojolicious 8.67+. 


Ubuntu 18.04 Bionic
- Mojolicious 7.59
- Mojolicious::Plugin::OpenAPI N/A

Ubuntu 20.04 Focal
- Mojolicious 8.33
- Mojolicious::Plugin::OpenAPI 2.21
    - JSON::Validator 3.15+
    - Mojolicious 8+

I don't know that Mojolicious::Plugin::OpenAPI 2.x can work with JSON::Validator 4.x... I see that with Koha on Ubuntu 20.04 we're using a JSON::Validator marked as 3.25+dfsg-1+koha3

Overall, I like Mojolicious, but I do not like this dependency hell.

-- 

It looks like Mojolicious::Plugin::OpenAPI 3.41 is compatible with either Mojolicious 8.0 or Mojolicious 8.67. Depends on if you trust the META.json/cpanfile or the Changes file more. It claims to work with JSON::Validator 4.11...
Comment 41 Mason James 2021-04-21 13:59:29 UTC
(In reply to Mason James from comment #37)
> (In reply to Bohdan from comment #36)
> > Hello. I've installed Koha 20.11.04 on latest Debian 10 and the API is
> > broken. I cannot get the list of available branches as it uses API.
> > 
> > Got the following in api error logs:
> > [ERROR] Warning: Could not load REST API spec bundle: Can't use string
> > ("parameters.json") as a HASH ref while "strict refs" in use at
> > /usr/share/perl5/JSON/Validator/OpenAPI/Mojolicious.pm line 272.

hi Bohdan, 
can you give more details?

1/ what is the url that causes the error?
2/ do you have any extra cpan modules installed?
3/ what version of the libjson-validator-perl package is installed?


the t/db_dependent/api/v1/libraries.t tests are passing for deb10/20.11.04, so it might be a problem specific to your system

https://jenkins.koha-community.org/view/20.11/job/Koha_20.11_D10/87
https://jenkins.koha-community.org/view/20.11/job/Koha_20.11_D10/87/consoleText
Comment 42 Bohdan 2021-04-23 10:32:29 UTC
(In reply to Mason James from comment #41)
> (In reply to Mason James from comment #37)
> > (In reply to Bohdan from comment #36)
> > > Hello. I've installed Koha 20.11.04 on latest Debian 10 and the API is
> > > broken. I cannot get the list of available branches as it uses API.
> > > 
> > > Got the following in api error logs:
> > > [ERROR] Warning: Could not load REST API spec bundle: Can't use string
> > > ("parameters.json") as a HASH ref while "strict refs" in use at
> > > /usr/share/perl5/JSON/Validator/OpenAPI/Mojolicious.pm line 272.
> 
> hi Bohdan, 
> can you give more details?
> 
> 1/ what is the url that causes the error?
> 2/ do you have any extra cpan modules installed?
> 3/ what version of the libjson-validator-perl package is installed?
> 
> 
> the t/db_dependent/api/v1/libraries.t tests are passing for deb10/20.11.04,
> so it might be a problem specific to your system
> 
> https://jenkins.koha-community.org/view/20.11/job/Koha_20.11_D10/87
> https://jenkins.koha-community.org/view/20.11/job/Koha_20.11_D10/87/
> consoleText

It seems I had Mojolicious installed via cpanm and apt. That caused the problem.
Thanks for the help.
Comment 43 Tomás Cohen Arazi 2022-02-28 11:03:55 UTC
After a day testing things, I'd say this is a duplicate for bug 30194. Please reopen if you don't agree.

*** This bug has been marked as a duplicate of bug 30194 ***