If I try to specify an external URL as an XSLT stylesheet it doesn't work. In the error logs I see messages like this: [2021/12/10 12:37:50] [WARN] XSLT::Base: Could not create file parser context for file "https://path/to/MARC21slim2intranetResults.xsl": No such file or directory at /kohadevbox/koha/Koha/XSLT/Base.pm line 330. I have verified that the .xsl file is accessible and that internal links (to MARC21slimUtils.xsl for instance) are valid too.
In my tests I get logs : 204 Exception::Class::Base::throw('Koha::Exceptions::Metadata::Invalid', 'id', 368, 'biblionumber', 369, 'format', 'marcxml', 'schema', 'MARC21', 'decoding _error', ':8: parser error : PCDATA invalid Char value 31^J <controlfield tag="001">00^_aD000015937</controlfield>...) called at /kohadevbox/koha/Koha/Bibl io/Metadata.pm line 73 205 Koha::Biblio::Metadata::record('Koha::Biblio::Metadata=HASH(0x55905c3efc40)') called at /kohadevbox/koha/Koha/Biblio.pm line 943 206 Koha::Biblio::get_marc_notes('Koha::Biblio=HASH(0x55905cbc1820)', 'HASH(0x55905cbbb5b0)') called at /kohadevbox/koha/catalogue/detail.pl line 148 207 eval {...} at /kohadevbox/koha/catalogue/detail.pl line 2 Very strange... Maybe some impact I can't explain of Bug 28734 adding transformMARCXML4XSLT() to get_marc_notes().
(In reply to Fridolin Somers from comment #1) > In my tests I get logs : > > 204 Exception::Class::Base::throw('Koha::Exceptions::Metadata::Invalid', > 'id', 368, 'biblionumber', 369, 'format', 'marcxml', 'schema', 'MARC21', Ah forget that, biblionumber=369 has bad MARCXML on purpose in KTD
(In reply to Owen Leonard from comment #0) > If I try to specify an external URL as an XSLT stylesheet it doesn't work. > In the error logs I see messages like this: > > [2021/12/10 12:37:50] [WARN] XSLT::Base: Could not create file parser > context for file "https://path/to/MARC21slim2intranetResults.xsl": No such > file or directory at /kohadevbox/koha/Koha/XSLT/Base.pm line 330. > > I have verified that the .xsl file is accessible and that internal links (to > MARC21slimUtils.xsl for instance) are valid too. Where did you specify it? Z3950/SRU target? If you have an external IntranetResults.xsl, how do you get to the slimUtils? I remember to have opened a bug report for includes over https. (Was not possible)
I think this was filed for the XSLT system preferences (results and detail pages), from the discussion on IRC. I've found bug 25216 - could this be the reason?
Bug 12758
I'm having the same problem, but I'm using http, not https. I copied MARC21slim2OPACResults.xsl, MARC21slim2OPACDetail.xsl, and MARC21slimUtils.xsl to http://dokument.space/scratchmonkey/[filename]; the only thing I changed was the pointer to the Utils file. No data appear. HOWEVER, when I use a detail file that I created for an instance running on 17.11, http://dokument.space/scratchmonkey/authcatDetail.xsl, it works like a charm.
I pulled an XSL from 17.11 repository, and I can't even get that to load. This really needs to be fixed. Our hands are tied as long as we can't make customizations.
(In reply to Christopher Brannon from comment #7) > I pulled an XSL from 17.11 repository, and I can't even get that to load. > This really needs to be fixed. Our hands are tied as long as we can't make > customizations. Could you check for any included files at the top of the XSLT? Newer XSLT also include one for language descriptions, so it's not only the Utils that needs to be changed/updated.
(In reply to Katrin Fischer from comment #8) > (In reply to Christopher Brannon from comment #7) > > I pulled an XSL from 17.11 repository, and I can't even get that to load. > > This really needs to be fixed. Our hands are tied as long as we can't make > > customizations. > > Could you check for any included files at the top of the XSLT? Newer XSLT > also include one for language descriptions, so it's not only the Utils that > needs to be changed/updated. Ah, and we also need to check the Utils itself isn#t asking for other included files.
(In reply to Katrin Fischer from comment #9) > (In reply to Katrin Fischer from comment #8) > > (In reply to Christopher Brannon from comment #7) > > > I pulled an XSL from 17.11 repository, and I can't even get that to load. > > > This really needs to be fixed. Our hands are tied as long as we can't make > > > customizations. > > > > Could you check for any included files at the top of the XSLT? Newer XSLT > > also include one for language descriptions, so it's not only the Utils that > > needs to be changed/updated. > > Ah, and we also need to check the Utils itself isn#t asking for other > included files. I'm not sure what you are asking for. I am new in the XSLT realm, so you would have to guide me to what you are looking for. I also don't have access to the backend, only what is available in the preferences. So if I have to look at core files, I can't.
Hm, it would have been really helpful to see the error from the logs (server side), because it probably has the information we need. Which files did you use that worked before?
(In reply to Katrin Fischer from comment #11) > Hm, it would have been really helpful to see the error from the logs (server > side), because it probably has the information we need. > > Which files did you use that worked before? Personally, we've only used the default settings, so we have not modified them yet. But if I do anything aside from default in the preferences, nothing works. It just goes blank.
If the XSLT is broken for whatever reason it will blank, but the logs will have the full error. The XSLT you link or use from elsewhere for sure needs to be able to work on it's own. In Koha's XSLT files there are includes for other files which is a good practice when working with XSLT, but it makes it harder to move them to a different directory. You need to make sure all files needed are in one place and paths corrected etc. When you can't access the error, this makes it harder of course. I'll try to give it a go and see if I can figure it out, but it will have to wait a little.
If Christopher's setup is similar to mine, I'm pretty sure the files he's using are MARC21slim2OPACDetail.xsl MARC21slim2OPACResults.xsl MARCslimUtils.xsl MARC21Languages.xsl OPACDetail and OPACResults both point to Utils, and Utils points to Languages. I can't verify it right now, but I think you might be able to fix this by adding the full external address (i.e. <xsl:include href="cheerfulvalleypl.online/somedirectory/MARC21Languages.xsl"/> at the top of the Utils file. The default location of the xsl stylesheets is /usr/share/koha/opac/htdocs/opac-tmpl/bootstrap/en/xslt If you can get SFTP root access to that directory, you can make changes. Give the original a different name first. In any case when you're editing xsl files, I highly recommend saving each incremental change with a new name before you try it. That way if you break the file (very easy to do) you revert to the latest stable version. I'm not sure what gets saved in the browser cache, but I've had more luck seeing changes if Koha uses a new xsl filename when it reloads a page.
I might have figured this out. I posted this to the Koha list to see if it works for someone other than me: You need to modify three files. Let's say you put them in in http://dokument.space/anew In opac_results.xml on line 10, change <xsl:import href="MARC21slimUtils.xsl"/> to <xsl:import href= http://dokument.space/anew"MARC21slimUtils.xsl"/> In opac_detail.xml do the same thing. In MARC21slimUtils.xsl at line 8, change <xsl:include href="MARC21Languages.xsl"/> to <xsl:include href="document.space/anew/MARC21Languages.xsl"/> You may need to clear your browser's cache or try it on a different computer. I really do have modified xml results and detail files in that location, in case anyone wants to look at the horrible things I've done to them as part of my rewrite of the MedStar Authors catalog.
I've essentially done this, although the ones I've tried to modify are MARC21Languages.xsl, MARC21slim2intranetDetail.xsl, and MARC21slimUtils.xsl. No joy.
I just cannot get an external sheet to work at all.
(In reply to Katrin Fischer from comment #13) > If the XSLT is broken for whatever reason it will blank, but the logs will > have the full error. > > The XSLT you link or use from elsewhere for sure needs to be able to work on > it's own. In Koha's XSLT files there are includes for other files which is a > good practice when working with XSLT, but it makes it harder to move them to > a different directory. You need to make sure all files needed are in one > place and paths corrected etc. > > When you can't access the error, this makes it harder of course. I'll try to > give it a go and see if I can figure it out, but it will have to wait a > little. It would be helpful to know which logs to look in. Can you point me to the right one? Also, I updated the paths Fred mentioned, but no luck. Is there more than one path in each file to update?
> It would be helpful to know which logs to look in. Can you point me to the > right one? Also, I updated the paths Fred mentioned, but no luck. Is there > more than one path in each file to update? You need to check all the <xsl:import statements in all files. Also make sure that your paths are correct. I am not sure which log, but if you do something like tail /var/log/koha/$KOHA_INSTANCE/*.log with your instance name, that should bring them all up.
It's possible this same thing was causing bug 37252 since we had one thing that's probably very rarely used which included MARC21slimUtils.xsl from the Library of Congress rather than our copy.
Log for that isn't particularly helpful, [WARN] XSLT::Base: I/O warning : failed to load external entity "https://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl" compilation error: file /kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/en/xslt/MARC21slim2MADS.xsl element include xsl:include : unable to load https://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl at /kohadevbox/koha/Koha/XSLT/Base.pm line 346. after two equally unhelpful [WARN] XSLT::Base: Unknown IO errorI/O warning : failed to load external entity "http://www.loc.gov/marcxml/xslt/MARC21slimUtils.xsl" and [WARN] XSLT::Base: Unknown IO errorI/O warning : failed to load external entity "http://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl" because the original loc.gov/marcxml returns a 301 to the http loc.gov/standards/marcxml and then that returns a 301 to https. Not getting very helpful debugging info from XSLT/Base.pm if it deals with a 301 by saying "Unknown IO errorI/O warning"
Hm, my guess would be that the URL got auto-guessed from the schema as we only have the file name in the include. Could that be? I tihnk we already had concluded that all includes needed to be changed to have the full URL of the external or file in Koha in order to work.
I didn't explain that very well. In https://git.koha-community.org/Koha-community/Koha/src/commit/f7e1559a7944abee1c1619cc1f1bbbd646622a60/koha-tmpl/intranet-tmpl/prog/en/xslt/MARC21slim2MADS.xsl#L9 we have the full URL to http://www.loc.gov/marcxml/xslt/MARC21slimUtils.xsl in an include. That fails with [WARN] XSLT::Base: Unknown IO errorI/O warning : failed to load external entity "http://www.loc.gov/marcxml/xslt/MARC21slimUtils.xsl" That URL actually returns a 301 to http://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl so if you alter the URL in the include to be that, it again fails with [WARN] XSLT::Base: Unknown IO errorI/O warning : failed to load external entity "http://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl" Then when you see that http://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl also redirects, to https at https://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl and change the URL in the include to that URL, you still get [WARN] XSLT::Base: I/O warning : failed to load external entity "https://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl" It's not a perfect test since the URL isn't available at both http and https, and maybe it falls under one of the cases that bug 12758 didn't cover, not sure.
I think that bug 23290 prevents external stylesheets from being imported/included in an XSLT. But I think you should be able to specify an external XSLT via the XSLT system preferences.
(In reply to David Cook from comment #24) > I think that bug 23290 prevents external stylesheets from being > imported/included in an XSLT. > > But I think you should be able to specify an external XSLT via the XSLT > system preferences. I think that's what they are trying to do in the first place, but it's not working.
(In reply to Phil Ringnalda from comment #23) > I didn't explain that very well. In > https://git.koha-community.org/Koha-community/Koha/src/commit/ > f7e1559a7944abee1c1619cc1f1bbbd646622a60/koha-tmpl/intranet-tmpl/prog/en/ > xslt/MARC21slim2MADS.xsl#L9 we have the full URL to > http://www.loc.gov/marcxml/xslt/MARC21slimUtils.xsl in an include. > > That fails with [WARN] XSLT::Base: Unknown IO errorI/O warning : failed to > load external entity "http://www.loc.gov/marcxml/xslt/MARC21slimUtils.xsl" > > That URL actually returns a 301 to > http://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl so if you > alter the URL in the include to be that, it again fails with [WARN] > XSLT::Base: Unknown IO errorI/O warning : failed to load external entity > "http://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl" > > Then when you see that > http://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl also > redirects, to https at > https://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl and change > the URL in the include to that URL, you still get [WARN] XSLT::Base: I/O > warning : failed to load external entity > "https://www.loc.gov/standards/marcxml/xslt/MARC21slimUtils.xsl" > > It's not a perfect test since the URL isn't available at both http and > https, and maybe it falls under one of the cases that bug 12758 didn't > cover, not sure. I didn't know about the existence of this file. Is this bug 37252 then?
(In reply to Katrin Fischer from comment #25) > I think that's what they are trying to do in the first place, but it's not > working. Since I've done a fair bit of work with the XSLTs, I'll take a quick look... -- http://localhost:8091/opac-tmpl/bootstrap/en/xslt/MARC21slim2OPACResults.xsl [2024/07/08 23:09:24] [WARN] http error : Operation in progress http error : Unknown IO error [2024/07/08 23:09:24] [WARN] XSLT::Base: Could not create file parser context for file "http://localhost:8091/opac-tmpl/bootstrap/en/xslt/MARC21slim2OPACResults.xsl": No such file or directory at /kohadevbox/koha/Koha/XSLT/Base.pm line 3 34. -- https://catalog.bywatersolutions.com/opac-tmpl/bootstrap/en/xslt/MARC21slim2OPACResults.xsl [2024/07/08 23:10:20] [WARN] XSLT::Base: I/O warning : failed to load external entity "MARC21slimUtils.xsl" compilation error: file unknown-5571f4c2a370 element import xsl:import : unable to load MARC21slimUtils.xsl at /kohadevbox/koha/Koha/XSLT/Base.pm line 346. -- I'll provide the explanations in another comment.
(In reply to David Cook from comment #27) > [2024/07/08 23:09:24] [WARN] http error : Operation in progress > http error : Unknown IO error > [2024/07/08 23:09:24] [WARN] XSLT::Base: Could not create file parser > context for file When using a HTTP URL, the URL is handed directly to the XSLT library, which is constrained by the security restrictions added by bug 23290. > [2024/07/08 23:10:20] [WARN] XSLT::Base: I/O warning : failed to load > external entity "MARC21slimUtils.xsl" > compilation error: file unknown-5571f4c2a370 element import > xsl:import : unable to load MARC21slimUtils.xsl > at /kohadevbox/koha/Koha/XSLT/Base.pm line 346. When using a HTTPS URL, the URL is first fetched by Koha code, and then presented to Koha as a string. Here it fails, probably because it can't find the file on the system. If loading a XSLT via HTTPS with properly set up filepaths, it should probably work. I'll see about doing a quick test over HTTPS...
(In reply to David Cook from comment #28) > If loading a XSLT via HTTPS with properly set up filepaths, it should > probably work. I'll see about doing a quick test over HTTPS... Indeed it does. I've created a /usr/share/koha/opac/htdocs/opac-tmpl/bootstrap/en/xslt/MARC21slim2OPACResults2.xsl and updated MARC21slimUtils.xsl to /usr/share/koha/opac/htdocs/opac-tmpl/bootstrap/en/xslt/MARC21slimUtils.xsl and it's working fine. Note that https://real.instance/opac-tmpl/bootstrap/en/xslt/MARC21slimUtils.xsl won't work here, because it's doing network activity, which is prohibited by bug 23290. -- We could potentially refine the security features from bug 23290, but that's the state of things at the moment. I think an argument could be made that we should fetch HTTP URLs the same way we do HTTPS URLs in the Koha code instead of the XSLT code, so that the external stylesheet works consistently in that way though.
Could people share their workflows for this? Are you uploading your own XSLTs to the same domain as Koha? To other domains you control? Are you referencing someone else's XLSTs? -- If we look at the "Using XML::LibXSLT::Security" section, we could modify our "read_net" callback to allow for reads if certain URL conditions are met. For instance, the URL's hostname/domain matches one we expect (probably in koha-conf.xml for security reasons - a global system preference could be manipulated by the same person updating the XSLT system preference which defeats the point of having the security at all). We could also check for file extension, which typically isn't a super robust check, but would help prevent network requests for other things.