Bug 37361 - REST API won't start properly without Internet access due to missing cache files
Summary: REST API won't start properly without Internet access due to missing cache files
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Packaging (show other bugs)
Version: 24.05
Hardware: All Linux
: P3 critical
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
: 36722 (view as bug list)
Depends on:
Blocks:
 
Reported: 2024-07-15 14:30 UTC by Andreas Tomiak
Modified: 2024-08-26 00:27 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: String patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Tomiak 2024-07-15 14:30:56 UTC
https://servername/api/v1/.html and any call to the API, even when done by Koha internally always shows error 404 or the error page.

starman/plack tries to contact external web servers for the schema files, as seen in plack-api-error.log:

[2024/07/15 14:47:45] [ERROR] Warning: Could not load REST API spec bundle: GET http://json-schema.org/draft-04/schema: Connection refused at /usr/share/perl5/JSON/Validator/Store.pm line 190.


The external http-access will require a proxy server and these are defined in /etc/profile.d/proxy.sh but not used by starman, which can be proved with

Window 1:
sudo tcpdump -A -i any 'host json-schema.org'

Window 2:
sudo koha-plack --restart koha-instance

the error messages will result.

The file environ of the starman processes in the linux process file system are empty.


Solution (dirty):

Source /etc/profile.d/proxy.sh with the definition of the proxies in /usr/sbin/koha-plack first.

The proxy environment variables will be used then and all will work as wanted.
Comment 1 Michalis Zisis 2024-08-14 20:17:55 UTC
Greetings,

i am facing the same issue in a similar case.

Likewise, the installation is connected to a restricted network, which is routed through a proxy. In my case, the proxy settings are defined in "/etc/environment". Moreover the library (due to being an old instance) has no active plack service.

After upgrading from 23.11 to 24.05 the same issue occurs, with the api address (https://servername/api/v1/) not responding to requests and returning a 404 error.

In "/var/log/koha/library/api-error.log" the same message appears "[ERROR] Warning: Could not load REST API spec bundle: GET http://json-schema.org/draft-04/schema: Connection refused at /usr/share /perl5/JSON/Validator/Store.pm line 190."

To verify, I tried connecting the server through an other network that does have any network restrictions, and koha functioned without an error.

So it seems that the api internally relies on retrieving the address "http://json-schema.org/draft-04/schema" and when this is not available, eg in the case of using a restricted network/proxy, there is an internal error that leads to generating a "404 not found" in the api.

In turn this causes errors in various sections, such as the patrons list failing to load, due to api returning the 404 response.

In regards for a possible solution, i would propose the api to not fail to run when it can't communicate in order to retrieve the needed (json) schema. Possibly, there could be a check to the api call in case of network error that would continue the api respose or some kind of fallback to a local storage of the schema/standards, so that even in case the network is not accessible, the api service would work.

Unfortunately, i do not have knowledge of the code the api service users in order to suggest a specific fix.

Thank you
Comment 2 David Cook 2024-08-15 01:31:31 UTC
Thanks for the report and the additional comment.

How have you installed Koha? And what OS are you using?

If you've used Debian packages, there should be a cached copy of the JSON schema on the server (probably at /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8), which should prevent Starman from trying to fetch http://json-schema.org/draft-04/schema

Recently, I did notice a particular version of libjson-validator-perl which was missing the cached file, so it might be related to that
Comment 3 David Cook 2024-08-15 02:30:04 UTC
As I note on bug 36722, it looks like libjson-validator-perl version 5.99+really5.14-koha2 does not provide the cached file that it should. 

I flagged this back in April, but it looks like I wasn't noisy enough about the problem...
Comment 4 David Cook 2024-08-15 02:30:28 UTC
*** Bug 36722 has been marked as a duplicate of this bug. ***
Comment 5 David Cook 2024-08-15 02:35:19 UTC
Noting that this is a packaging bug because it's due to the Koha community packaging of libjson-validator-perl
Comment 6 Andreas Tomiak 2024-08-15 10:58:58 UTC
Hello,

I´ve installed it on Debian 11 with the proposed Koah sources. It was upgraded several times. The problem appears in April, so this may be a hint to a solution.

The cache displays only these files:

ls -la /usr/share/perl5/JSON/Validator/cache/
total 8
drwxr-xr-x 2 root root 4096 Jun 12 10:04 .
drwxr-xr-x 4 root root 4096 Jun 12 10:04 ..
lrwxrwxrwx 1 root root   58 Apr  3 14:16 36d1bd12eeed51e86c8695bd8876a9df -> ../../../../openapi-specification/schemas/v2.0/schema.json
lrwxrwxrwx 1 root root   58 Apr  3 14:16 a516498b60c53096b2ce2cd83ebe0abc -> ../../../../openapi-specification/schemas/v3.0/schema.json
Comment 7 Andreas Tomiak 2024-08-15 11:22:59 UTC
Some more information:

dpkg-query --list | grep libjson-validator-perl
ii  libjson-validator-perl                 5.99+really5.14-koha2              all          module to validate data against a JSON schema

The links are correct:

xxx@yyy:/usr/share/perl5/JSON/Validator/cache$ ls -la ../../../../openapi-specification/schemas/v2.0/schema.json
-rw-r--r-- 1 root root 40247 Feb 16  2021 ../../../../openapi-specification/schemas/v2.0/schema.json
xxx@yyy:/usr/share/perl5/JSON/Validator/cache$ ls -la ../../../../openapi-specification/schemas/v3.0/schema.json
-rw-r--r-- 1 root root 35456 Feb 16  2021 ../../../../openapi-specification/schemas/v3.0/schema.json

Our installation sources:

cat /etc/apt/sources.list.d/koha.list 
deb [signed-by=/usr/share/keyrings/koha-keyring.gpg] http://debian.koha-community.org/koha stable main bullseye

I´ve changed rights on /etc/apt/sources.list.d/koha.list and /etc/apt/apt.conf.d/ from 600 to 644 now. Wrong umask setting due to a weird security theatre here :-(
Comment 8 David Cook 2024-08-15 23:28:18 UTC
(In reply to Andreas Tomiak from comment #7)
> Some more information:
> 
> dpkg-query --list | grep libjson-validator-perl
> ii  libjson-validator-perl                 5.99+really5.14-koha2            
> all          module to validate data against a JSON schema

Yeah, that's a bad package. I'm trying to get the package maintainer to fix it. 

An "apt upgrade" or "apt-get upgrade" would've updated the libjson-validator-perl from the previously working version to this bad version.

As a short term measure, you could try downgrading this package, although 24.05 might need the newer one. I haven't tried out 24.05 yet, so I'm not sure.

But working on an update to the cached files provided with this package...
Comment 9 Mason James 2024-08-19 13:37:37 UTC
(In reply to David Cook from comment #8)
> (In reply to Andreas Tomiak from comment #7)
> > Some more information:
> > 
> > dpkg-query --list | grep libjson-validator-perl
> > ii  libjson-validator-perl                 5.99+really5.14-koha2            
> > all          module to validate data against a JSON schema
> 
> Yeah, that's a bad package. I'm trying to get the package maintainer to fix
> it. 
> 

i've added a corrected package to the koha-staging repo, for CI testing

the package includes the missing cached schema files in 
 /usr/share/perl5/JSON/Validator/cache

the package is located at...
 https://debian.koha-community.org/koha-staging/pool/main/libj/libjson-validator-perl/libjson-validator-perl_5.99+really5.14-koha3_all.deb
Comment 10 Andreas Tomiak 2024-08-19 14:18:00 UTC
Hello David,

I´ve installed the debian package, removed my patch and restarted koha-common.

The API now works as expected.

Thanks a lot, fine work.


Andreas
Comment 11 David Cook 2024-08-20 01:31:15 UTC
(In reply to Mason James from comment #9)
> i've added a corrected package to the koha-staging repo, for CI testing
> 
> the package includes the missing cached schema files in 
>  /usr/share/perl5/JSON/Validator/cache
> 
> the package is located at...
>  https://debian.koha-community.org/koha-staging/pool/main/libj/libjson-
> validator-perl/libjson-validator-perl_5.99+really5.14-koha3_all.deb

Thanks, Mason. I really appreciate it.

I figure once it goes through the CI testing we'll get it out on the stable apt repo?
Comment 12 Mason James 2024-08-20 04:36:42 UTC
(In reply to David Cook from comment #11)
> (In reply to Mason James from comment #9)
> > i've added a corrected package to the koha-staging repo, for CI testing
> > 
> > the package includes the missing cached schema files in 
> >  /usr/share/perl5/JSON/Validator/cache
> > 
> > the package is located at...
> >  https://debian.koha-community.org/koha-staging/pool/main/libj/libjson-
> > validator-perl/libjson-validator-perl_5.99+really5.14-koha3_all.deb
> 
> Thanks, Mason. I really appreciate it.
> 
> I figure once it goes through the CI testing we'll get it out on the stable
> apt repo?

yes! hopefully will push today after next jenkins build
Comment 13 Mason James 2024-08-22 23:24:28 UTC
> > I figure once it goes through the CI testing we'll get it out on the stable
> > apt repo?
> 
> yes! hopefully will push today after next jenkins build

jenkins build is passing, so i've pushed this package live to 24.05/stable repo

$ apt policy  libjson-validator-perl
libjson-validator-perl:
  Installed: 5.99+really5.14-koha3
  Candidate: 5.99+really5.14-koha3
  Version table:
 *** 5.99+really5.14-koha3 500
        500 http://debian.koha-community.org/koha stable/main amd64 Packages
        100 /var/lib/dpkg/status
     5.14+dfsg-1 500
        500 http://ftp.nz.debian.org/debian bookworm/main amd64 Packages
Comment 14 David Cook 2024-08-22 23:34:23 UTC
(In reply to Mason James from comment #13)
> jenkins build is passing, so i've pushed this package live to 24.05/stable
> repo

Huzzah!

I think we can probably close this report then?