This is arguably the same issue as bug 23653 except that it's a different website this time. Also, Koha won't crash if json-schema.org is down, but the API will return a 404 for all endpoints. Like bug 23653, I think that we should use a local copy of the specification. At the moment, the way to do that is to download http://json-schema.org/draft-04/schema#, save it into our source tree, and then run the following: my $schema = JSON::Validator::Schema::OpenAPIv2->new; $schema->specification($saved_schema_spec); In theory, that's it. I'll look at providing a patch shortly...
Note also that JSON::Validator::Schema::Draft4 used by JSON::Validator::Schema::OpenAPIv2 is set to http://json-schema.org/draft-04/schema# which redirects to the HTTPS version, so every worker does 2 web requests to it on startup. Yikes...
Note an easy way to prove this is to run the following: Window 1: sudo tcpdump -A -i any 'host json-schema.org' Window 2: sudo koha-plack --restart kohadev
Looks like JSON::Validator isn't caching files unless JSON_VALIDATOR_CACHE_PATH is defined... It did cache when I changed koha-plack to do this: if JSON_VALIDATOR_CACHE_PATH=/opt/cache ${STARMAN} ${STARMANOPTS}; then
And now I'm extra confused... because JSON::Validator::Schema::OpenAPIv2 has a specification of http://swagger.io/v2/schema.json but http://json-schema.org/draft-04/schema# is what's being downloaded... I suppose because the $schema for http://swagger.io/v2/schema.json is http://json-schema.org/draft-04/schema#
If I force JSON::Validator::Store into debug mode, it does look like it's downloading it during the resolving process... [JSON::Validator] Resolve curr: ref=http://json-schema.org/draft-04/schema#/properties/description,base_url=http://swagger.io/v2/schema.json fragment= id=http://swagger.io/v2/schema.json root=HASH(0x55d20f974060) schema=HASH(0x55d20f9740 60) source=load [JSON::Validator] Resolve state: base_url=http://json-schema.org/draft-04/schema fragment=/definitions/stringArray id=http://json-schema.org/draft-04/schema root=HASH(0x55d2057e4a40) schema=HASH(0x55d20f702f40) source=schema/base_url I'm starting to think that bug 30194 actually made no difference here... -- [JSON::Validator] Load from cache /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df [JSON::Validator] Load from URL http://json-schema.org/draft-04/schema And Koha can't add json-schema.org to the cache, because Koha doesn't have write access to /usr/share/perl5/JSON/Validator/cache... 36d1bd12eeed51e86c8695bd8876a9df is the cached version of the http://swagger.io/v2/schema.json file.
Note that /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df and api/swagger-v2-schema.json are the same...
I think the shortest solution for this one is going to be to add the contents of http://json-schema.org/draft-04/schema# as /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8 And that can be done via the libjson-validator-perl package that the Koha community puts together.
Changing this to a "Packaging" bug I guess. Technically, we could point the cache paths elsewhere and provide our own cached files but happy enough to be continuing as we are...
At some point I did ```shell ktd --shell sudo vim /usr/share/perl5/JSON/Validator/Schema/Draft4.pm ``` and replaced `http` with `https`on line 12. And it started working. But I'm not sure it is really it, as I cannot make it fail anymore :-P Just noting in case it is useful. I think the solution is on packaging as well.
When I reported and analyzed this one, I wasn't having any errors. I just wanted to make things faster/more efficient. The error for me was hypothetical only, so that's interesting that you did experience the error! To make it reliably fail, you can add the following to your /etc/hosts: 127.0.0.1 json-schema.org -- But yeah we should be able to fix this one by packaging (or by changing the cache paths and providing our own cached files).
(In reply to David Cook from comment #7) > I think the shortest solution for this one is going to be to add the > contents of http://json-schema.org/draft-04/schema# as > /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8 > > And that can be done via the libjson-validator-perl package that the Koha > community puts together. /usr/share/perl5/JSON/Validator/Store.pm removes the fragment from the URL, so you end up with the following: echo -n "http://json-schema.org/draft-04/schema" | md5sum 49c95b866e40f788892a7fb3c816b0e8 -
Ok now I'm super curious... My Koha 23.11 dev system on Ubuntu 22.04 libjson-validator-perl 5.99+really5.08-koha1 /usr/share/perl5/JSON/Validator/cache/089e74a6d17f64af17a9efd6d0fa0de6 /usr/share/perl5/JSON/Validator/cache/10a5eeb37fcd5d829449028f7ceb0774 /usr/share/perl5/JSON/Validator/cache/33912dbbde6e1d936140f1c82b283d01 /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df /usr/share/perl5/JSON/Validator/cache/3be3f46eb248daf48925640f8ef057e8 /usr/share/perl5/JSON/Validator/cache/3d35aac549d951f4cf9182ff47bff0b4 /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8 /usr/share/perl5/JSON/Validator/cache/4a31fe43be9e23ca9eb8d9e9faba8892 /usr/share/perl5/JSON/Validator/cache/546acf85ddc442761c18517490215b90 /usr/share/perl5/JSON/Validator/cache/630949337805585c8e52deea27d11419 /usr/share/perl5/JSON/Validator/cache/7fe97ed1a4c3fac607dd276b2b298275 /usr/share/perl5/JSON/Validator/cache/a0f5b4b4e75ea17fc09e88ec0343d148 /usr/share/perl5/JSON/Validator/cache/a516498b60c53096b2ce2cd83ebe0abc /usr/share/perl5/JSON/Validator/cache/c6f188eb288cf986f23db49297b25e83 /usr/share/perl5/JSON/Validator/cache/d18065ce8fb1f748e766b2737bae5200 /usr/share/perl5/JSON/Validator/cache/d8cf7ae7a0fd14accadf5d18bc84d14f /usr/share/perl5/JSON/Validator/cache/ea34d47d4e060a1c3b12d2287aff89a7 /usr/share/perl5/JSON/Validator/cache/eaa832720f36cff0abc20c05236a9cd9 My Koha main dev system on Debian 12 libjson-validator-perl 5.99+really5.14-koha1 /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df /usr/share/perl5/JSON/Validator/cache/a516498b60c53096b2ce2cd83ebe0abc
So not a problem in prod for people using the "koha" apt repo. But it is a problem in the "koha-staging" apt repo.
Ok and now it looks like it's a problem in "koha"! dpkg -l libjson-validator-perl Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-======================-=====================-============-============================================= ii libjson-validator-perl 5.99+really5.14-koha2 all module to validate data against a JSON schema dpkg -L libjson-validator-perl /. /usr /usr/share /usr/share/doc /usr/share/doc/libjson-validator-perl /usr/share/doc/libjson-validator-perl/NEWS.Debian.gz /usr/share/doc/libjson-validator-perl/changelog.Debian.gz /usr/share/doc/libjson-validator-perl/changelog.gz /usr/share/doc/libjson-validator-perl/copyright /usr/share/man /usr/share/man/man3 /usr/share/man/man3/JSON::Validator.3pm.gz /usr/share/man/man3/JSON::Validator::Error.3pm.gz /usr/share/man/man3/JSON::Validator::Formats.3pm.gz /usr/share/man/man3/JSON::Validator::Joi.3pm.gz /usr/share/man/man3/JSON::Validator::Schema.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft201909.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft4.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft6.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft7.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::OpenAPIv2.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::OpenAPIv3.3pm.gz /usr/share/man/man3/JSON::Validator::Store.3pm.gz /usr/share/man/man3/JSON::Validator::URI.3pm.gz /usr/share/man/man3/JSON::Validator::Util.3pm.gz /usr/share/perl5 /usr/share/perl5/JSON /usr/share/perl5/JSON/Validator /usr/share/perl5/JSON/Validator/Error.pm /usr/share/perl5/JSON/Validator/Formats.pm /usr/share/perl5/JSON/Validator/Joi.pm /usr/share/perl5/JSON/Validator/Schema /usr/share/perl5/JSON/Validator/Schema/Draft201909.pm /usr/share/perl5/JSON/Validator/Schema/Draft4.pm /usr/share/perl5/JSON/Validator/Schema/Draft6.pm /usr/share/perl5/JSON/Validator/Schema/Draft7.pm /usr/share/perl5/JSON/Validator/Schema/OpenAPIv2.pm /usr/share/perl5/JSON/Validator/Schema/OpenAPIv3.pm /usr/share/perl5/JSON/Validator/Schema.pm /usr/share/perl5/JSON/Validator/Store.pm /usr/share/perl5/JSON/Validator/URI.pm /usr/share/perl5/JSON/Validator/Util.pm /usr/share/perl5/JSON/Validator/cache /usr/share/perl5/JSON/Validator.pm /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df /usr/share/perl5/JSON/Validator/cache/a516498b60c53096b2ce2cd83ebe0abc
It looks like my warning wasn't heeded and now it's affecting at least 2 people in prod... *** This bug has been marked as a duplicate of bug 37361 ***