Summary: | Koha not caching http://json-schema.org/draft-04/schema# | ||
---|---|---|---|
Product: | Koha | Reporter: | David Cook <dcook> |
Component: | Packaging | Assignee: | Bugs List <koha-bugs> |
Status: | RESOLVED DUPLICATE | QA Contact: | Testopia <testopia> |
Severity: | normal | ||
Priority: | P5 - low | CC: | mtj, tomascohen |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
See Also: |
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36546 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 |
||
Change sponsored?: | --- | Patch complexity: | --- |
Documentation contact: | Documentation submission: | ||
Text to go in the release notes: | Version(s) released in: | ||
Circulation function: |
Description
David Cook
2024-04-30 02:46:39 UTC
Note also that JSON::Validator::Schema::Draft4 used by JSON::Validator::Schema::OpenAPIv2 is set to http://json-schema.org/draft-04/schema# which redirects to the HTTPS version, so every worker does 2 web requests to it on startup. Yikes... Note an easy way to prove this is to run the following: Window 1: sudo tcpdump -A -i any 'host json-schema.org' Window 2: sudo koha-plack --restart kohadev Looks like JSON::Validator isn't caching files unless JSON_VALIDATOR_CACHE_PATH is defined... It did cache when I changed koha-plack to do this: if JSON_VALIDATOR_CACHE_PATH=/opt/cache ${STARMAN} ${STARMANOPTS}; then And now I'm extra confused... because JSON::Validator::Schema::OpenAPIv2 has a specification of http://swagger.io/v2/schema.json but http://json-schema.org/draft-04/schema# is what's being downloaded... I suppose because the $schema for http://swagger.io/v2/schema.json is http://json-schema.org/draft-04/schema# If I force JSON::Validator::Store into debug mode, it does look like it's downloading it during the resolving process... [JSON::Validator] Resolve curr: ref=http://json-schema.org/draft-04/schema#/properties/description,base_url=http://swagger.io/v2/schema.json fragment= id=http://swagger.io/v2/schema.json root=HASH(0x55d20f974060) schema=HASH(0x55d20f9740 60) source=load [JSON::Validator] Resolve state: base_url=http://json-schema.org/draft-04/schema fragment=/definitions/stringArray id=http://json-schema.org/draft-04/schema root=HASH(0x55d2057e4a40) schema=HASH(0x55d20f702f40) source=schema/base_url I'm starting to think that bug 30194 actually made no difference here... -- [JSON::Validator] Load from cache /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df [JSON::Validator] Load from URL http://json-schema.org/draft-04/schema And Koha can't add json-schema.org to the cache, because Koha doesn't have write access to /usr/share/perl5/JSON/Validator/cache... 36d1bd12eeed51e86c8695bd8876a9df is the cached version of the http://swagger.io/v2/schema.json file. Note that /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df and api/swagger-v2-schema.json are the same... I think the shortest solution for this one is going to be to add the contents of http://json-schema.org/draft-04/schema# as /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8 And that can be done via the libjson-validator-perl package that the Koha community puts together. Changing this to a "Packaging" bug I guess. Technically, we could point the cache paths elsewhere and provide our own cached files but happy enough to be continuing as we are... At some point I did ```shell ktd --shell sudo vim /usr/share/perl5/JSON/Validator/Schema/Draft4.pm ``` and replaced `http` with `https`on line 12. And it started working. But I'm not sure it is really it, as I cannot make it fail anymore :-P Just noting in case it is useful. I think the solution is on packaging as well. When I reported and analyzed this one, I wasn't having any errors. I just wanted to make things faster/more efficient. The error for me was hypothetical only, so that's interesting that you did experience the error! To make it reliably fail, you can add the following to your /etc/hosts: 127.0.0.1 json-schema.org -- But yeah we should be able to fix this one by packaging (or by changing the cache paths and providing our own cached files). (In reply to David Cook from comment #7) > I think the shortest solution for this one is going to be to add the > contents of http://json-schema.org/draft-04/schema# as > /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8 > > And that can be done via the libjson-validator-perl package that the Koha > community puts together. /usr/share/perl5/JSON/Validator/Store.pm removes the fragment from the URL, so you end up with the following: echo -n "http://json-schema.org/draft-04/schema" | md5sum 49c95b866e40f788892a7fb3c816b0e8 - Ok now I'm super curious... My Koha 23.11 dev system on Ubuntu 22.04 libjson-validator-perl 5.99+really5.08-koha1 /usr/share/perl5/JSON/Validator/cache/089e74a6d17f64af17a9efd6d0fa0de6 /usr/share/perl5/JSON/Validator/cache/10a5eeb37fcd5d829449028f7ceb0774 /usr/share/perl5/JSON/Validator/cache/33912dbbde6e1d936140f1c82b283d01 /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df /usr/share/perl5/JSON/Validator/cache/3be3f46eb248daf48925640f8ef057e8 /usr/share/perl5/JSON/Validator/cache/3d35aac549d951f4cf9182ff47bff0b4 /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8 /usr/share/perl5/JSON/Validator/cache/4a31fe43be9e23ca9eb8d9e9faba8892 /usr/share/perl5/JSON/Validator/cache/546acf85ddc442761c18517490215b90 /usr/share/perl5/JSON/Validator/cache/630949337805585c8e52deea27d11419 /usr/share/perl5/JSON/Validator/cache/7fe97ed1a4c3fac607dd276b2b298275 /usr/share/perl5/JSON/Validator/cache/a0f5b4b4e75ea17fc09e88ec0343d148 /usr/share/perl5/JSON/Validator/cache/a516498b60c53096b2ce2cd83ebe0abc /usr/share/perl5/JSON/Validator/cache/c6f188eb288cf986f23db49297b25e83 /usr/share/perl5/JSON/Validator/cache/d18065ce8fb1f748e766b2737bae5200 /usr/share/perl5/JSON/Validator/cache/d8cf7ae7a0fd14accadf5d18bc84d14f /usr/share/perl5/JSON/Validator/cache/ea34d47d4e060a1c3b12d2287aff89a7 /usr/share/perl5/JSON/Validator/cache/eaa832720f36cff0abc20c05236a9cd9 My Koha main dev system on Debian 12 libjson-validator-perl 5.99+really5.14-koha1 /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df /usr/share/perl5/JSON/Validator/cache/a516498b60c53096b2ce2cd83ebe0abc So not a problem in prod for people using the "koha" apt repo. But it is a problem in the "koha-staging" apt repo. Ok and now it looks like it's a problem in "koha"! dpkg -l libjson-validator-perl Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-======================-=====================-============-============================================= ii libjson-validator-perl 5.99+really5.14-koha2 all module to validate data against a JSON schema dpkg -L libjson-validator-perl /. /usr /usr/share /usr/share/doc /usr/share/doc/libjson-validator-perl /usr/share/doc/libjson-validator-perl/NEWS.Debian.gz /usr/share/doc/libjson-validator-perl/changelog.Debian.gz /usr/share/doc/libjson-validator-perl/changelog.gz /usr/share/doc/libjson-validator-perl/copyright /usr/share/man /usr/share/man/man3 /usr/share/man/man3/JSON::Validator.3pm.gz /usr/share/man/man3/JSON::Validator::Error.3pm.gz /usr/share/man/man3/JSON::Validator::Formats.3pm.gz /usr/share/man/man3/JSON::Validator::Joi.3pm.gz /usr/share/man/man3/JSON::Validator::Schema.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft201909.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft4.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft6.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft7.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::OpenAPIv2.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::OpenAPIv3.3pm.gz /usr/share/man/man3/JSON::Validator::Store.3pm.gz /usr/share/man/man3/JSON::Validator::URI.3pm.gz /usr/share/man/man3/JSON::Validator::Util.3pm.gz /usr/share/perl5 /usr/share/perl5/JSON /usr/share/perl5/JSON/Validator /usr/share/perl5/JSON/Validator/Error.pm /usr/share/perl5/JSON/Validator/Formats.pm /usr/share/perl5/JSON/Validator/Joi.pm /usr/share/perl5/JSON/Validator/Schema /usr/share/perl5/JSON/Validator/Schema/Draft201909.pm /usr/share/perl5/JSON/Validator/Schema/Draft4.pm /usr/share/perl5/JSON/Validator/Schema/Draft6.pm /usr/share/perl5/JSON/Validator/Schema/Draft7.pm /usr/share/perl5/JSON/Validator/Schema/OpenAPIv2.pm /usr/share/perl5/JSON/Validator/Schema/OpenAPIv3.pm /usr/share/perl5/JSON/Validator/Schema.pm /usr/share/perl5/JSON/Validator/Store.pm /usr/share/perl5/JSON/Validator/URI.pm /usr/share/perl5/JSON/Validator/Util.pm /usr/share/perl5/JSON/Validator/cache /usr/share/perl5/JSON/Validator.pm /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df /usr/share/perl5/JSON/Validator/cache/a516498b60c53096b2ce2cd83ebe0abc It looks like my warning wasn't heeded and now it's affecting at least 2 people in prod... *** This bug has been marked as a duplicate of bug 37361 *** |