https://jenkins.koha-community.org/job/Koha_Master_D8/lastCompletedBuild/console # Failed test 'ListIdentifiers with resumptionToken 2' # at t/db_dependent/OAI/Server.t line 153. # Structures begin differing at: # $got->{ListIdentifiers}{header}[2]{datestamp} = '2018-02-13T17:46:23Z' # $expected->{ListIdentifiers}{header}[2]{datestamp} = '2018-02-13T17:46:22Z' # PARAM:--- # resumptionToken: marcxml/6/1970-01-01T00:00:00Z/2018-02-13T23:59:59Z//0/0 # verb: ListIdentifiers # EXPECTED:--- # ListIdentifiers: # header: # - datestamp: 2018-02-13T17:46:22Z # identifier: TEST:686 # - datestamp: 2018-02-13T17:46:22Z # identifier: TEST:687 # - datestamp: 2018-02-13T17:46:22Z # identifier: TEST:688 # resumptionToken: # content: marcxml/9/1970-01-01T00:00:00Z/2018-02-13T23:59:59Z//0/0 # cursor: 9 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # RESPONSE:--- # ListIdentifiers: # header: # - datestamp: 2018-02-13T17:46:22Z # identifier: TEST:686 # - datestamp: 2018-02-13T17:46:22Z # identifier: TEST:687 # - datestamp: 2018-02-13T17:46:23Z # identifier: TEST:688 # resumptionToken: # content: marcxml/9/1970-01-01T00:00:00Z/2018-02-13T23:59:59Z//0/0 # cursor: 9 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # Failed test 'ListRecords marcxml with resumptionToken 2' # at t/db_dependent/OAI/Server.t line 153. # Structures begin differing at: # $got->{ListRecords}{record}[2]{header}{datestamp} = '2018-02-13T17:46:23Z' # $expected->{ListRecords}{record}[2]{header}{datestamp} = '2018-02-13T17:46:22Z' # PARAM:--- # resumptionToken: marcxml/6/1970-01-01T00:00:00Z/2018-02-13T23:59:59Z//0/0 # verb: ListRecords # EXPECTED:--- # ListRecords: # record: # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:686 # metadata: # record: # controlfield: # content: ' lng' # tag: 008 # datafield: # - ind1: ' ' # ind2: ' ' # subfield: # code: a # content: Title 6 # tag: 245 # - ind1: ' ' # ind2: ' ' # subfield: # - code: c # content: 686 # - code: d # content: 586 # tag: 999 # leader: 00126 a2200061 4500 # xmlns: http://www.loc.gov/MARC21/slim # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:687 # metadata: # record: # controlfield: # content: ' lng' # tag: 008 # datafield: # - ind1: ' ' # ind2: ' ' # subfield: # code: a # content: Title 7 # tag: 245 # - ind1: ' ' # ind2: ' ' # subfield: # - code: c # content: 687 # - code: d # content: 587 # tag: 999 # leader: 00126 a2200061 4500 # xmlns: http://www.loc.gov/MARC21/slim # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:688 # metadata: # record: # controlfield: # content: ' lng' # tag: 008 # datafield: # - ind1: ' ' # ind2: ' ' # subfield: # code: a # content: Title 8 # tag: 245 # - ind1: ' ' # ind2: ' ' # subfield: # - code: c # content: 688 # - code: d # content: 588 # tag: 999 # leader: 00126 a2200061 4500 # xmlns: http://www.loc.gov/MARC21/slim # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # resumptionToken: # content: marcxml/9/1970-01-01T00:00:00Z/2018-02-13T23:59:59Z//0/0 # cursor: 9 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # RESPONSE:--- # ListRecords: # record: # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:686 # metadata: # record: # controlfield: # content: ' lng' # tag: 008 # datafield: # - ind1: ' ' # ind2: ' ' # subfield: # code: a # content: Title 6 # tag: 245 # - ind1: ' ' # ind2: ' ' # subfield: # - code: c # content: 686 # - code: d # content: 586 # tag: 999 # leader: 00126 a2200061 4500 # xmlns: http://www.loc.gov/MARC21/slim # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:687 # metadata: # record: # controlfield: # content: ' lng' # tag: 008 # datafield: # - ind1: ' ' # ind2: ' ' # subfield: # code: a # content: Title 7 # tag: 245 # - ind1: ' ' # ind2: ' ' # subfield: # - code: c # content: 687 # - code: d # content: 587 # tag: 999 # leader: 00126 a2200061 4500 # xmlns: http://www.loc.gov/MARC21/slim # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # - header: # datestamp: 2018-02-13T17:46:23Z # identifier: TEST:688 # metadata: # record: # controlfield: # content: ' lng' # tag: 008 # datafield: # - ind1: ' ' # ind2: ' ' # subfield: # code: a # content: Title 8 # tag: 245 # - ind1: ' ' # ind2: ' ' # subfield: # - code: c # content: 688 # - code: d # content: 588 # tag: 999 # leader: 00126 a2200061 4500 # xmlns: http://www.loc.gov/MARC21/slim # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd # resumptionToken: # content: marcxml/9/1970-01-01T00:00:00Z/2018-02-13T23:59:59Z//0/0 # cursor: 9 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # Failed test 'ListRecords oai_dc with resumptionToken 2' # at t/db_dependent/OAI/Server.t line 153. # Structures begin differing at: # $got->{ListRecords}{record}[2]{header}{datestamp} = '2018-02-13T17:46:23Z' # $expected->{ListRecords}{record}[2]{header}{datestamp} = '2018-02-13T17:46:22Z' # PARAM:--- # resumptionToken: oai_dc/6/1970-01-01T00:00:00Z/2018-02-13T23:59:59Z//0/0 # verb: ListRecords # EXPECTED:--- # ListRecords: # record: # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:686 # metadata: # oai_dc:dc: # dc:language: lng # dc:title: Title 6 # dc:type: {} # xmlns:dc: http://purl.org/dc/elements/1.1/ # xmlns:oai_dc: http://www.openarchives.org/OAI/2.0/oai_dc/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:687 # metadata: # oai_dc:dc: # dc:language: lng # dc:title: Title 7 # dc:type: {} # xmlns:dc: http://purl.org/dc/elements/1.1/ # xmlns:oai_dc: http://www.openarchives.org/OAI/2.0/oai_dc/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:688 # metadata: # oai_dc:dc: # dc:language: lng # dc:title: Title 8 # dc:type: {} # xmlns:dc: http://purl.org/dc/elements/1.1/ # xmlns:oai_dc: http://www.openarchives.org/OAI/2.0/oai_dc/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd # resumptionToken: # content: oai_dc/9/1970-01-01T00:00:00Z/2018-02-13T23:59:59Z//0/0 # cursor: 9 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # RESPONSE:--- # ListRecords: # record: # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:686 # metadata: # oai_dc:dc: # dc:language: lng # dc:title: Title 6 # dc:type: {} # xmlns:dc: http://purl.org/dc/elements/1.1/ # xmlns:oai_dc: http://www.openarchives.org/OAI/2.0/oai_dc/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd # - header: # datestamp: 2018-02-13T17:46:22Z # identifier: TEST:687 # metadata: # oai_dc:dc: # dc:language: lng # dc:title: Title 7 # dc:type: {} # xmlns:dc: http://purl.org/dc/elements/1.1/ # xmlns:oai_dc: http://www.openarchives.org/OAI/2.0/oai_dc/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd # - header: # datestamp: 2018-02-13T17:46:23Z # identifier: TEST:688 # metadata: # oai_dc:dc: # dc:language: lng # dc:title: Title 8 # dc:type: {} # xmlns:dc: http://purl.org/dc/elements/1.1/ # xmlns:oai_dc: http://www.openarchives.org/OAI/2.0/oai_dc/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd # resumptionToken: # content: oai_dc/9/1970-01-01T00:00:00Z/2018-02-13T23:59:59Z//0/0 # cursor: 9 # request: http://localhost # xmlns: http://www.openarchives.org/OAI/2.0/ # xmlns:xsi: http://www.w3.org/2001/XMLSchema-instance # xsi:schemaLocation: http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd # Looks like you failed 3 tests of 29. [17:46:24] t/db_dependent/OAI/Server.t ....................................
I did not manage to mock DateTime->now
https://jenkins.koha-community.org/job/Koha_Master_D8/363/ https://jenkins.koha-community.org/job/Koha_Master_D8/367/ https://jenkins.koha-community.org/job/Koha_17.05_D8/78/
Created attachment 71673 [details] [review] Bug 20191: Make OAI/Server.t tests use timestamps from the DB Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
+ Koha::Biblio::Metadatas->search({ biblionumber => $biblionumber }) Should use ->find instead, with (biblionumber, 'marcxml', 'MARC21'), do not you think?
Created attachment 71816 [details] [review] Bug 20191: Use find with unique key combination To prevent unexpected issues later.
Pushed to master, thanks Tomas!
still fails: https://jenkins.koha-community.org/job/Koha_Master_D8/377/console
and again: https://jenkins.koha-community.org/job/Koha_Master_D8/381/console
I tried with Time::Fake but failed.
Created attachment 72009 [details] [review] Bug 20191: Try and fix random failure from Server.t - v3...
This is my try with Time::Fake, what's wrong?
I use `stress -c 4` to recreate the failures.
These tests are constantly failing, I am going to remove them once and for all if these random failures are not fixed.
(In reply to Jonathan Druart from comment #14) > These tests are constantly failing, I am going to remove them once and for > all if these random failures are not fixed. The HTTP::OAI::Header grabs gmtime() which could change between the creation time and the call time... How about this impending counter patch?
Created attachment 73669 [details] [review] Bug 20191: Use Test::MockTime to hold time still The problem is the calls to HTTP::OAI::Header, etc. may reference gmtime which is continually changing by the second. By forcing time to lock for all the tests, except the last one, we can be assured things should not fail. TEST PLAN --------- install libtest-mocktime-perl apply only this patch restart_all in kshell, prove t/db_dependent/OAI/Server.t run koha qa test tools
Created attachment 73684 [details] [review] Bug 20191: Use Test::MockTime to hold time still The problem is the calls to HTTP::OAI::Header, etc. may reference gmtime which is continually changing by the second. By forcing time to lock for all the tests, except the last one, we can be assured things should not fail. TEST PLAN --------- install libtest-mocktime-perl apply the patch restart_all in kshell, prove t/db_dependent/OAI/Server.t run koha qa test tools Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Created attachment 73685 [details] [review] Bug 20191: (QA follow-up) Require Jessie's Test::MockTime Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
I tested with Mark's patch using 'stress -c 4' on a 2 cores kohadevbox (cpu load above 3). I managed to reproduce the issue without the patch, and can confirm the patch deals with the problem correctly. The solution is the right one too, one we talked about but didn't find the spot in which environment was leaking inside the tests. Good catch Mark!
Excellent Mark! It works great thanks a lot! Note for later: Maybe we should replace our mock of DateTime->now with this new module. Pushed to master for 18.05