Actually OAI-PMH in Koha uses HTTP::OAI lib from version in Jessie 3.27. This module is not compatible with version 4.03 which is in Ubuntu Xenial and Debian Stretch. You get the error : Can't call method "start_element" on an undefined value at /usr/share/perl5/HTTP/OAI/Response.pm line 114 This is because the method HTTP::OAI::Response::generate() has changed, it needs an argument $driver.
See https://packages.debian.org/search?searchon=names&keywords=libhttp-oai-perl
Related: https://lists.katipo.co.nz/public/koha/2016-April/045226.html https://rt.cpan.org/Public/Bug/Display.html?id=102010#txn-1569001
*** Bug 19732 has been marked as a duplicate of this bug. ***
[Koha-devel] State of HTTP::OAI versions http://lists.koha-community.org/pipermail/koha-devel/2018-February/044302.html
I have uploaded libhttp-oai-perl_4.06-1~koha1_all.deb to the unstable repository. Does that help?
(In reply to Mirko Tietgen from comment #5) > I have uploaded libhttp-oai-perl_4.06-1~koha1_all.deb to the unstable > repository. Does that help? Better wait we solve some issues. I'm in talks with the maintainer to fix some standing issues. The fix for xslt is oh their repo but there hasn't been a release yet.
Ok, I have removed the file again.
Created attachment 75331 [details] [review] Bug 17704: Adapt Koha::OAI::* to the new HTTP::OAI api This is my attempt to adapt the existing implementation to the new version of the HTTP::OAI lib. Currently 4.06 is released, but 4.07 should be used once released, as it contains the fix for the ->xslt method missing.
1. Is this backwards compatible to 3.27? 2. Does it obsolete Bug 20437?
(In reply to Mirko Tietgen from comment #9) > 1. Is this backwards compatible to 3.27? No. > 2. Does it obsolete Bug 20437? Absolutely not. This is WIP and is not functional. I just dropped the code for other to take over. If we make it work, we will still need a new version of HTTP::OAI released (with the fixes for xslt and maybe others) and package it for older distros. So no. This is not production ready at all. And 18.05 should use 3.27, and I'd say supported stable releases of Koha too, to support Stretch and newer Ubuntus.
Looks like HTTP::OAI 4.07 is now released.
(In reply to Ere Maijala (away 13Jul-30Aug) from comment #11) > Looks like HTTP::OAI 4.07 is now released. I tried 4.10 today and it still doesn't work. #NOTE: I think this one is related to a customization I did so maybe ignore it /cgi-bin/koha/oai.pl?verb=Identify Can't locate object method "requestURL" via package "Koha::OAI::Server::Identify" at .../lib/Koha/OAI/Server/Repository.pm line 170 This one should be relevant though: Can't call method "processing_instruction" on an undefined value at /usr/local/share/perl/5.28.1/HTTP/OAI/Response.pm line 115
As Tomas mentioned in https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17704#c8, we have to update our usage of HTTP::OAI in Koha in order to use newer versions, and when we do use a newer version, it'll need to be > 4.07. Note: It's not enough to just use 4.07. I can prove that Koha master will fail with 4.10 in koha-testing-docker: 1) cpanm HTTP::OAI This should install 4.10 2) restart_all 3) Enable OAI-PMH at http://localhost:8081/cgi-bin/koha/admin/preferences.pl 4) Go to http://localhost:8080/cgi-bin/koha/oai.pl 5) Note "Internal Server Error" in the browser 6) Note "Can't call method "processing_instruction" on an undefined value at /usr/local/share/perl/5.24.1/HTTP/OAI/Response.pm line 115." in plack-error.log
(In reply to David Cook from comment #12) > Can't call method "processing_instruction" on an undefined value > at /usr/local/share/perl/5.28.1/HTTP/OAI/Response.pm line 115 HTTP::OAI::Response::generate() used in Koha/OAI/Server/Repository.pm requires a $driver argument in 4.10 (and 4.07)... So some type of patch is needed.
(In reply to Tomás Cohen Arazi from comment #10) > This is WIP and is not functional. I just dropped the code > for > other to take over. Ok, I'll make a note to myself to take a look at this again. It looks like you're probably nearly there?
Created attachment 107214 [details] [review] Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07)
Created attachment 107215 [details] [review] Bug 17704: Adapt Koha::OAI::* to the new HTTP::OAI api This is my attempt to adapt the existing implementation to the new version of the HTTP::OAI lib. Currently 4.06 is released, but 4.07 should be used once released, as it contains the fix for the ->xslt method missing. Signed-off-by: David Cook <dcook@prosentient.com.au>
Created attachment 107216 [details] [review] Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07) Signed-off-by: David Cook <dcook@prosentient.com.au>
Putting this one to "In Discussion" instead of "Signed off" for the moment, since we need to handle the HTTP::OAI dependency first before we make changes to this code...
Comment on attachment 75331 [details] [review] Bug 17704: Adapt Koha::OAI::* to the new HTTP::OAI api Review of attachment 75331 [details] [review]: ----------------------------------------------------------------- ::: Koha/OAI/Server/Repository.pm @@ +152,5 @@ > + Handler => my $builder = XML::LibXML::SAX::Builder->new() > + ); > + > + $self->{ driver } = $driver; > + $self->{ builder } = $builder; Tomas, did you put driver and builder here for a specific purpose? I'm guessing for testing purposes? I don't see any functional reason otherwise? @@ +156,5 @@ > + $self->{ builder } = $builder; > + > + $driver->start_oai_pmh(); > + > + $response->set_handler($w); I don't understand the purpose of this, but I noticed it in the HTTP::OAI::Repository example, so I imagine it must be necessary. I find HTTP::OAI to be an unnecessarily arcane library, and since it's just a couple lines here, it's probably not worth taking the time to interrogate it too closely...
(In reply to David Cook from comment #18) > Created attachment 107216 [details] [review] [review] > Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07) > > Signed-off-by: David Cook <dcook@prosentient.com.au> I am getting same error, <pre>Can't call method "processing_instruction" on an undefined value at /usr/local/share/perl/5.30.0/HTTP/OAI/Response.pm line 115. koha version 20.11.12 HTTP::OAI 4.11 to add this patch in my file there is no lines like this : $response->set_handler($w); #$response->xslt( "/opac-tmpl/xslt/OAI.xslt" ); $response->generate($driver); lines in my file (Koha/OAI/Server/Repository.pm): $response->set_handler( XML::SAX::Writer->new( Output => *STDOUT ) ); $response->xslt( "/opac-tmpl/xslt/OAI.xslt" ); $response->generate; bless $self, $class; return $self;
We should probably do a patch for this at some point so that we don't have to package the older HTTP::OAI modules.
Created attachment 141762 [details] [review] Bug 17704: Fix OAI breakage when using HTTP::OAI 4.03+ bump version in cpanfile, and remove from debian/list-deps (no test plan)
(In reply to David Cook from comment #22) > We should probably do a patch for this at some point so that we don't have > to package the older HTTP::OAI modules. patch is tested as working for me using HTTP::OAI 4.11
Created attachment 141763 [details] pic
Created attachment 141764 [details] [review] Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07) Signed-off-by: David Cook <dcook@prosentient.com.au> Signed-off-by: Mason James <mtj@kohaaloha.com>
Created attachment 141765 [details] [review] Bug 17704: Bump version in cpanfile
Created attachment 141766 [details] [review] Bug 17704: Adapt Koha::OAI::* to the new HTTP::OAI api This is my attempt to adapt the existing implementation to the new version of the HTTP::OAI lib. Currently 4.06 is released, but 4.07 should be used once released, as it contains the fix for the ->xslt method missing. Signed-off-by: David Cook <dcook@prosentient.com.au> Signed-off-by: Mason James <mtj@kohaaloha.com>
3 patches, 3 different authors, picking first.
Applying: Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07) error: sha1 information is lacking or useless (Koha/OAI/Server/Repository.pm). error: could not build fake ancestor Patch failed at 0001 Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07)
Created attachment 146457 [details] [review] Bug 17704: Adapt Koha::OAI::* to the new HTTP::OAI api This is my attempt to adapt the existing implementation to the new version of the HTTP::OAI lib. Currently 4.06 is released, but 4.07 should be used once released, as it contains the fix for the ->xslt method missing. Signed-off-by: David Cook <dcook@prosentient.com.au> Signed-off-by: Mason James <mtj@kohaaloha.com>
Created attachment 146458 [details] [review] Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07) Signed-off-by: David Cook <dcook@prosentient.com.au> Signed-off-by: Mason James <mtj@kohaaloha.com>
Created attachment 146459 [details] [review] Bug 17704: Bump version in cpanfile Signed-off-by: Mason James <mtj@kohaaloha.com>
(In reply to Kyle M Hall from comment #30) > Applying: Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07) > error: sha1 information is lacking or useless > (Koha/OAI/Server/Repository.pm). > error: could not build fake ancestor > Patch failed at 0001 Bug 17704: Enable ->xslt method (must use HTTP::OAI >= > 4.07) oops, the order of the 3 patches got mixed up i've uploaded them again in the correct order ----------------------------------------- mason@xen1:~/g/k/m$ git log | head -1 commit 6ac569453b4b66dac7e221a4244869ca3cee3268 mason@xen1:~/g/k/m$ git gz apply -s 17704 fatal: Resolve operation not in progress, we are not resuming. fatal: No rebase in progress? Bug 17704 - Fix OAI breakage when using HTTP::OAI 4.03+ 146457 - Bug 17704: Adapt Koha::OAI::* to the new HTTP::OAI api 146458 - Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07) 146459 - Bug 17704: Bump version in cpanfile Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 17704: Adapt Koha::OAI::* to the new HTTP::OAI api Applying: Bug 17704: Enable ->xslt method (must use HTTP::OAI >= 4.07) Applying: Bug 17704: Bump version in cpanfile
> ----------------------------------------- > mason@xen1:~/g/k/m$ git log | head -1 > commit 6ac569453b4b66dac7e221a4244869ca3cee3268 > oops, typo thats... mason@xen1:~/g/k/m$ git log | head -1 commit 32258f82b5dd6a2bef3faedf06211eaf674f674c
Unit test failures. Please let me know if it just that I'm doing something wrong. I install HTTP::OAI 4.12 before running the tests. t/db_dependent/OAI/Server.t ... 13/34 # Failed test 'ListMetadataFormats' # at t/db_dependent/OAI/Server.t line 184. # Comparing hash keys of $data # Missing: 'request' # PARAM:--- # verb: ListMetadataFormats # EXPECTED:--- # ListMetadataFormats: # metadataFormat: # - metadataNamespace: http://www.openarchives.org/OAI/2.0/oai_dc/ # metadataPrefix: oai_dc # schema: http://www.openarchives.org/OAI/2.0/oai_dc.xsd # - metadataNamespace: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim # metadataPrefix: marc21 # schema: http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # - metadataNamespace: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim # metadataPrefix: marcxml # schema: http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # RESPONSE:--- # ListMetadataFormats: # metadataFormat: # - metadataNamespace: http://www.openarchives.org/OAI/2.0/oai_dc/ # metadataPrefix: oai_dc # schema: http://www.openarchives.org/OAI/2.0/oai_dc.xsd # - metadataNamespace: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim # metadataPrefix: marc21 # schema: http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # - metadataNamespace: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim # metadataPrefix: marcxml # schema: http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # Failed test 'ListIdentifiers without metadataPrefix' # at t/db_dependent/OAI/Server.t line 184. # Comparing hash keys of $data # Missing: 'error', 'request' # PARAM:--- # verb: ListIdentifiers # EXPECTED:--- # error: # code: badArgument # content: Required argument 'metadataPrefix' was undefined # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # RESPONSE:--- # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # Failed test 'ListIdentifiers' # at t/db_dependent/OAI/Server.t line 184. # Comparing hash keys of $data # Missing: 'request' # PARAM:--- # metadataPrefix: marcxml # verb: ListIdentifiers # EXPECTED:--- # ListIdentifiers: # header: # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:483 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:484 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:485 # resumptionToken: # content: !!perl/hash:Test::Deep::Regexp # val: !!perl/regexp (?^u:^marcxml/3////0/0/\d+$) # cursor: 3 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # RESPONSE:--- # ListIdentifiers: # header: # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:483 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:484 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:485 # resumptionToken: # content: marcxml/3////0/0/486 # cursor: '3' # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # Failed test 'ListIdentifiers' # at t/db_dependent/OAI/Server.t line 184. # Comparing hash keys of $data # Missing: 'request' # PARAM:--- # metadataPrefix: marcxml # verb: ListIdentifiers # EXPECTED:--- # ListIdentifiers: # header: # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:483 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:484 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:485 # resumptionToken: # content: !!perl/hash:Test::Deep::Regexp # val: !!perl/regexp (?^u:^marcxml/3////0/0/\d+$) # cursor: 3 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # RESPONSE:--- # ListIdentifiers: # header: # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:483 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:484 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:485 # resumptionToken: # content: marcxml/3////0/0/486 # cursor: '3' # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # Failed test 'ListIdentifiers with resumptionToken 1' # at t/db_dependent/OAI/Server.t line 184. # Comparing hash keys of $data # Missing: 'request' # PARAM:--- # resumptionToken: marcxml/3/1970-01-01T00:00:00Z/2023-03-10T23:59:59Z//0/0/486 # verb: ListIdentifiers # EXPECTED:--- # ListIdentifiers: # header: # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:486 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:487 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:488 # resumptionToken: # content: !!perl/hash:Test::Deep::Regexp # val: !!perl/regexp (?^u:^marcxml/6/1970-01-01T00:00:00Z/2023-03-10T23:59:59Z//0/0/\d+$) # cursor: 6 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # RESPONSE:--- # ListIdentifiers: # header: # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:486 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:487 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:488 # resumptionToken: # content: marcxml/6/1970-01-01T00:00:00Z/2023-03-10T23:59:59Z//0/0/489 # cursor: '6' # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # Failed test 'ListIdentifiers with resumptionToken 2' # at t/db_dependent/OAI/Server.t line 184. # Comparing hash keys of $data # Missing: 'request' # PARAM:--- # resumptionToken: marcxml/6/1970-01-01T00:00:00Z/2023-03-10T23:59:59Z//0/0/489 # verb: ListIdentifiers # EXPECTED:--- # ListIdentifiers: # header: # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:489 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:490 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:491 # resumptionToken: # content: !!perl/hash:Test::Deep::Regexp # val: !!perl/regexp (?^u:^marcxml/9/1970-01-01T00:00:00Z/2023-03-10T23:59:59Z//0/0/\d+$) # cursor: 9 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # RESPONSE:--- # ListIdentifiers: # header: # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:489 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:490 # - datestamp: 2023-03-10T13:04:26Z # identifier: TEST:491 # resumptionToken: # content: marcxml/9/1970-01-01T00:00:00Z/2023-03-10T23:59:59Z//0/0/492 # cursor: '9' # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd Can't locate object method "generate" via package "marcxml/9/1970-01-01T00:00:00Z/2023-03-10T23:59:59Z//0/0/492" (perhaps you forgot to load "marcxml/9/1970-01-01T00:00:00Z/2023-03-10T23:59:59Z//0/0/492"?) at /usr/local/share/perl/5.32.1/HTTP/OAI/PartialList.pm line 30. # Looks like your test exited with 11 just after 18. t/db_dependent/OAI/Server.t ... Dubious, test returned 11 (wstat 2816, 0xb00) Failed 22/34 subtests t/db_dependent/OAI/Sets.t ..... ok Test Summary Report ------------------- t/db_dependent/OAI/Server.t (Wstat: 2816 Tests: 18 Failed: 6) Failed tests: 13-18 Non-zero exit status: 11 Parse errors: Bad plan. You planned 34 tests but ran 18. Files=3, Tests=175, 3 wallclock secs ( 0.02 usr 0.00 sys + 2.44 cusr 0.21 csys = 2.67 CPU) Result: FAIL
(In reply to Kyle M Hall from comment #36) > Unit test failures. Please let me know if it just that I'm doing something > wrong. I install HTTP::OAI 4.12 before running the tests. hi Kyle oops, i forgot to run 'prove' after testing this it seems the newer version of HTTP::OAI has changed its behavior somewhat the tests look to some 'updating' before they will pass on 4.12
Should this be failed QA?
It seems so. Switching to it so it's clearer that is rescuable and not blocked due to uncertainty about a fix.
Are we still working on this?
(In reply to Martin Renvoize from comment #40) > Are we still working on this? I keep meaning to look at this again... but low on the list...