Bug 8300 - Koha should have mechanized testing suite
Summary: Koha should have mechanized testing suite
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Jared Camins-Esakov
QA Contact: Marcel de Rooy
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-06-23 20:41 UTC by Jared Camins-Esakov
Modified: 2013-12-05 20:09 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 8300: Add mechanized unit test for batch import (7.53 KB, patch)
2012-06-23 20:54 UTC, Jared Camins-Esakov
Details | Diff | Splinter Review
Bug 8300: Add mechanized unit test for batch import (9.00 KB, patch)
2012-09-10 10:32 UTC, Jared Camins-Esakov
Details | Diff | Splinter Review
Bug 8300: Add mechanized unit test for batch import (9.24 KB, patch)
2012-09-14 12:59 UTC, Jared Camins-Esakov
Details | Diff | Splinter Review
Bug 8300: Add mechanized unit test for batch import (9.57 KB, patch)
2012-09-14 13:27 UTC, Magnus Enger
Details | Diff | Splinter Review
Patch for new dependency (889 bytes, patch)
2012-10-12 12:35 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Screenshot showing the link followed by the test script on line 157 (17.42 KB, image/png)
2012-10-18 08:24 UTC, Magnus Enger
Details
Bug 8300 - Add Test::WWW::Mechanize as an optional dependency (926 bytes, patch)
2013-03-20 13:26 UTC, Magnus Enger
Details | Diff | Splinter Review
Bug 8300: Add mechanized unit test for batch import (9.64 KB, patch)
2013-04-12 15:08 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 8300 - Add Test::WWW::Mechanize as an optional dependency (1001 bytes, patch)
2013-04-12 15:08 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 8300: Add mechanized unit test for batch import (9.75 KB, patch)
2013-04-15 08:26 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 8300 - Add Test::WWW::Mechanize as an optional dependency (1.02 KB, patch)
2013-04-15 08:27 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jared Camins-Esakov 2012-06-23 20:41:14 UTC
There should be a set of tests that use Test::WWW::Mechanize to automatically test the functionality of Koha.
Comment 1 Jared Camins-Esakov 2012-06-23 20:54:26 UTC Comment hidden (obsolete)
Comment 2 Jonathan Druart 2012-08-20 12:30:53 UTC
I got an error when I launch tests:

% prove t/db_dependent/www/batch.t
t/db_dependent/www/batch.t .. 14/19 
#   Failed test 'found book'
#   at t/db_dependent/www/batch.t line 111.
#     searched: "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Trans"...
#   can't find: "Data structures : Esakov, Jeffrey"
#         LCSS: "cture"
# LCSS context: "  \x{0a}    \x{0a}\x{09}<li><a href="/cgi-bin/koha/tools/picture-"


Test::WWW::Mechanize seems to be a new dependency.
Comment 3 Jared Camins-Esakov 2012-08-29 19:54:37 UTC
(In reply to comment #2)
> I got an error when I launch tests:
> 
> % prove t/db_dependent/www/batch.t
> t/db_dependent/www/batch.t .. 14/19 
> #   Failed test 'found book'
> #   at t/db_dependent/www/batch.t line 111.
> #     searched: "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Trans"...
> #   can't find: "Data structures : Esakov, Jeffrey"
> #         LCSS: "cture"
> # LCSS context: "  \x{0a}    \x{0a}\x{09}<li><a
> href="/cgi-bin/koha/tools/picture-"
> 
> 
> Test::WWW::Mechanize seems to be a new dependency.

I've been struggling to figure out the problem, and I finally realized- my sample record is MARC21. I'll update the test to have both UNIMARC and MARC21.
Comment 4 Jared Camins-Esakov 2012-09-10 10:32:16 UTC Comment hidden (obsolete)
Comment 5 Magnus Enger 2012-09-14 06:57:41 UTC
After applying the patch I get this:

$ prove -v t/db_dependent/www/batch.t
t/db_dependent/www/batch.t .. 
1..20
ok 1 - connect to intranet
ok 2 - login to staff client
ok 3 - load main page
Link not found at t/db_dependent/www/batch.t line 63.
# Looks like you planned 20 tests but ran 3.
# Looks like your test exited with 255 just after 3.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 17/20 subtests 

Test Summary Report
-------------------
t/db_dependent/www/batch.t (Wstat: 65280 Tests: 3 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 20 tests but ran 3.
Files=1, Tests=3,  4 wallclock secs ( 0.04 usr  0.00 sys +  0.38 cusr  0.04 csys =  0.46 CPU)
Result: FAIL

I think the problem is that after the new look of the staff client was introduced, the link to tools no longer contains the text "Tools". Changing line 63 of t/db_dependent/www/batch.t to this: 

$agent->follow_link_ok( { url_regex => qr/tools-home/i }, 'open tools module' );

makes the test succeed.
Comment 6 Magnus Enger 2012-09-14 08:15:28 UTC
The environment variables that need to be set should probably be documented somewhere too? Not sure where though. A README in t/db_dependent/www/ perhaps?
Comment 7 Magnus Enger 2012-09-14 08:53:35 UTC
This would be more of an enhancement request, but I imagine lots of tests could require lots of different env variables, to point the scripts to users, records etc that could be used for testing. 

I can also imagine wanting to run the tests against a number of different installations, both on and off localhost. 

Would it make sense to have these tests read config files that contain all the different settings required to test a given installation, and then point the tests to the right config with a single env var?
Comment 8 Jared Camins-Esakov 2012-09-14 12:59:11 UTC Comment hidden (obsolete)
Comment 9 Magnus Enger 2012-09-14 13:27:00 UTC Comment hidden (obsolete)
Comment 10 Magnus Enger 2012-09-14 13:28:45 UTC
Not tested on a UNIMARC setup, but still signing off. It's not as if this will break anything if there really is an error for UNIMARC.
Comment 11 Mason James 2012-09-19 05:06:04 UTC
> 
> To test:
> 1. Install Test::WWW::Mechanize
> > sudo apt-get install libtest-www-mechanize-perl

can we get a follow-up patch to add www-mech as a dependancy package?
Comment 12 Marcel de Rooy 2012-10-12 12:35:45 UTC Comment hidden (obsolete)
Comment 13 Marcel de Rooy 2012-10-12 12:42:38 UTC
My test (on current master) results in:

t/db_dependent/www/batch.t .. 14/20
#   Failed test 'found book'
#   at t/db_dependent/www/batch.t line 165.
#     searched: "\x{0a}\x{0a}<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Tra"...
#   can't find: "Data structures"
#         LCSS: "Data str"
# LCSS context: "" rel="gb_page_center[600,500]">Englewood Data str"
t/db_dependent/www/batch.t .. 19/20 # Looks like you failed 1 test of 20.
t/db_dependent/www/batch.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/20 subtests

Test Summary Report
-------------------
t/db_dependent/www/batch.t (Wstat: 256 Tests: 20 Failed: 1)
  Failed test:  15
  Non-zero exit status: 1
Files=1, Tests=20, 14 wallclock secs ( 0.02 usr  0.01 sys +  0.51 cusr  0.02 csys =  0.56 CPU)
Result: FAIL
Comment 14 Marcel de Rooy 2012-10-12 12:47:50 UTC
Just looking at the previous comment, I wonder why it should find the title Data structures. On Manage staged MARC records, I can see the file name, but not this title.
Please explain. Changing status to reflect need of clarification. 
Thank you.
Comment 15 Magnus Enger 2012-10-17 08:23:09 UTC
I have started to collect some ideas here: http://wiki.koha-community.org/wiki/Mechanized_testing_suite_RFC
Comment 16 Magnus Enger 2012-10-17 09:01:18 UTC
(In reply to comment #13)
> t/db_dependent/www/batch.t .. 14/20
> #   Failed test 'found book'
> #   at t/db_dependent/www/batch.t line 165.
> #     searched: "\x{0a}\x{0a}<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
> Tra"...
> #   can't find: "Data structures"
> #         LCSS: "Data str"
> # LCSS context: "" rel="gb_page_center[600,500]">Englewood Data str"
> t/db_dependent/www/batch.t .. 19/20 # Looks like you failed 1 test of 20.
> t/db_dependent/www/batch.t .. Dubious, test returned 1 (wstat 256, 0x100)
> Failed 1/20 subtests

What MARC-flavour did you test with, Marcel? I just ran the tests on current master without any problems:

$ prove -v t/db_dependent/www/batch.t
t/db_dependent/www/batch.t .. 
1..20
ok 1 - connect to intranet
ok 2 - login to staff client
ok 3 - load main page
ok 4 - open tools module
ok 5 - go to stage MARC
ok 6 - uploaded file
ok 7 - upload succeeded
ok 8 - reopen stage MARC page
ok 9 - stage MARC
ok 10 - have job ID
ok 11 - job was completed
ok 12 - reopen stage MARC page at end of upload
ok 13 - stage MARC
ok 14 - view batch
ok 15 - found book
ok 16 - imported records into catalog
ok 17 - successfully imported record
ok 18 - revert import
ok 19 - getting reverted bib
ok 20 - bib is gone
ok
All tests successful.
Files=1, Tests=20, 25 wallclock secs ( 0.04 usr  0.00 sys +  0.56 cusr  0.04 csys =  0.64 CPU)
Result: PASS
Comment 17 Marcel de Rooy 2012-10-17 11:27:48 UTC
(In reply to comment #16)
> What MARC-flavour did you test with, Marcel? I just ran the tests on current
> master without any problems:

MARC21
Comment 18 Magnus Enger 2012-10-18 08:24:20 UTC
Created attachment 12899 [details]
Screenshot showing the link followed by the test script on line 157
Comment 19 Magnus Enger 2012-10-18 08:50:39 UTC
(In reply to comment #14)
> Just looking at the previous comment, I wonder why it should find the title
> Data structures. On Manage staged MARC records, I can see the file name, but
> not this title.
> Please explain. Changing status to reflect need of clarification. 
> Thank you.

I think I found an axplanation for this: After the file is uploaded you get a list of information as shown in the screenshot I just uploaded. This list has a link labelled "Manage staged records" which takes you directly to the records you just uploaded. This is the link the test follows on line 157:

$agent->follow_link_ok( { text => 'Manage staged records' }, 'view batch' );

And this page should display the titles of the uploaded/staged records. From the test output it looks like your title might have been truncated to ">Englewood Data str"? 

If you want to spend more time on this you might try and put an __END__ below line 165, like so:

$agent->content_contains( $bookdescription, 'found book' );
__END__
$agent->form_number(5);

This will let you visit "Staged MARC record management" and see what the results of the staging looks like (before it is removed further down in the test).

(I just tested the patch on another installation and it still works for me :-)
Comment 20 Marcel de Rooy 2012-10-18 10:01:13 UTC
I still do not get this working.
Manually importing the data file, I have the impression that this file is the actual problem. When importing, I see under Citation: Englewood Data str
This probably should have been Data structures.
Is this marc datafile really in good shape? I do not think so.
Comment 21 Magnus Enger 2012-10-18 10:12:28 UTC
That *is* weird. For me, both the test itself and staging and importing t/db_dependent/www/data/marc21record.mrc manually works as expected. But the format of the marc file definitely seems to be the problem then.
Comment 22 Jonathan Druart 2012-11-22 13:21:50 UTC
The floor routine is provided by the POSIX module which is not loaded.
Comment 23 Magnus Enger 2013-03-18 16:49:39 UTC
I just had a go at this again. 

- The first patch still applies, and for me all the tests pass (on a NORMARC setup). POSIX not being loaded does not seem to be an issue. 

- The second patch does not apply because new dependencies have been added since the path was made. Making a new patch for this should be easy. 

How do we move this bug forwards? It seems to work well for me but not for Marcel...
Comment 24 Magnus Enger 2013-03-20 13:26:14 UTC Comment hidden (obsolete)
Comment 25 Magnus Enger 2013-03-20 13:42:12 UTC
I just staged and imported the record in the marc21record.mrc file successfully into Koha, and the results look good to me. Not sure that file is the cause of Marcel's problems...

I'm resetting this to "Needs signoff" so we can hopefully get another set of eyes on it and rule out any weirdness in either my setup or that of Marcel.
Comment 26 Kyle M Hall 2013-04-12 15:08:07 UTC Comment hidden (obsolete)
Comment 27 Kyle M Hall 2013-04-12 15:08:19 UTC Comment hidden (obsolete)
Comment 28 Jonathan Druart 2013-04-15 08:25:00 UTC
QA comment:
Nice unit test.
Tests pass here (UNIMARC install), so I mark these patches as Passed QA.

ps: I maintain my previous remark: we should load the POSIX module for the floor routine. My SO patch will contain this modification.
Comment 29 Jonathan Druart 2013-04-15 08:26:53 UTC
Created attachment 17437 [details] [review]
Bug 8300: Add mechanized unit test for batch import

This new unit test confirms that importing records via the Stage
MARC import tool still works by connecting to a Koha instance and
importing a record then reverting the import, checking at each step
of the way that everything is as it should be.

To test:
1. Install Test::WWW::Mechanize
> sudo apt-get install libtest-www-mechanize-perl
2. Set environment variables to reflect your Koha instance:
> export KOHA_USER=kohaadmin
> export KOHA_PASS=katikoan
> export KOHA_INTRANET_URL=http://localhost:8080
> export KOHA_OPAC_URL=http://localhost
3. Run the test:
> prove t/db_dependent/www/batch.t

This updated patch now handles both MARC21 and UNIMARC installations.

Signed-off-by: Magnus Enger <magnus@enger.priv.no>
All tests pass with marcflavour = MARC21 and NORMARC. Test #15 fails
when marcflavour = UNIMARC, but from what I can see, that is because
I'm testing on a MARC21 setup, missing the UNIMARC frameworks etc.
I'm signing off - QA folks, please ask for a proper UNIMARC signoff
if you feel it is needed.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Amended patch: Load the POSIX module (for the floor routine).
Comment 30 Jonathan Druart 2013-04-15 08:27:26 UTC
Created attachment 17438 [details] [review]
Bug 8300 - Add Test::WWW::Mechanize as an optional dependency

This patch replaces an earlier patch by Marcel de Rooy, which
had become outdated because lots of new dependencies were
added since the patch was made.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Comment 31 Jared Camins-Esakov 2013-05-01 11:57:38 UTC
This does not work with Plack, of course, since importing records doesn't work under Plack, but I am pushing the patch anyway as it could be useful for people who use Apache.
Comment 32 Jared Camins-Esakov 2013-05-01 11:57:48 UTC
This patch has been pushed to master and 3.12.x.