Bug 20505 - Selenium tests fail if server not running
Summary: Selenium tests fail if server not running
Status: CLOSED WONTFIX
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: master
Hardware: All All
: P5 - low minor (vote)
Assignee: Mark Tompsett
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-04-01 14:41 UTC by Mark Tompsett
Modified: 2023-12-28 20:42 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 20505: Patch selenium tests to skip gracefully (2.39 KB, patch)
2018-04-01 14:50 UTC, Mark Tompsett
Details | Diff | Splinter Review
Bug 20505: Patch selenium tests to skip gracefully (2.44 KB, patch)
2018-04-17 16:55 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 20505: Patch selenium tests to skip gracefully (2.54 KB, patch)
2018-05-04 09:27 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 20505: Friendlier death message (2.24 KB, patch)
2018-05-07 16:03 UTC, Mark Tompsett
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Tompsett 2018-04-01 14:41:48 UTC
If the dependencies for the Selenium tests are installed, but the steps recommended on https://wiki.koha-community.org/wiki/Using_Selenium_with_Koha are not taken before running the selenium tests, they die non-gracefully. Patch them to skip informatively.
Comment 1 Mark Tompsett 2018-04-01 14:50:23 UTC
Created attachment 73517 [details] [review]
Bug 20505: Patch selenium tests to skip gracefully

TEST PLAN
---------
1) In a koha shell on kohadevbox, run
    prove -v t/db_dependent/selenium
   until it doesn't skip because of missing
   dependencies, but rather because of not running
   a selenium server.
2) apply this patch and restart_all
3) rerun the tests
   -- they will skip gracefully and informatively.
4) run koha qa test tools
Comment 2 Nick Clemens 2018-04-17 16:55:23 UTC
Created attachment 74364 [details] [review]
Bug 20505: Patch selenium tests to skip gracefully

TEST PLAN
---------
1) In a koha shell on kohadevbox, run
    prove -v t/db_dependent/selenium
   until it doesn't skip because of missing
   dependencies, but rather because of not running
   a selenium server.
2) apply this patch and restart_all
3) rerun the tests
   -- they will skip gracefully and informatively.
4) run koha qa test tools

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 3 Nick Clemens 2018-04-17 16:56:00 UTC
To trigger the test plan I installed the dependencies but did not setup the server
Comment 4 Marcel de Rooy 2018-05-04 09:27:41 UTC
Created attachment 75054 [details] [review]
Bug 20505: Patch selenium tests to skip gracefully

TEST PLAN
---------
1) In a koha shell on kohadevbox, run
    prove -v t/db_dependent/selenium
   until it doesn't skip because of missing
   dependencies, but rather because of not running
   a selenium server.
2) apply this patch and restart_all
3) rerun the tests
   -- they will skip gracefully and informatively.
4) run koha qa test tools

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 5 Jonathan Druart 2018-05-04 12:27:32 UTC
I do not know if this is valid, I would prefer to explode if the server is not running and to know there is something wrong in the setup.

The error is explicit: "Selenium server did not return proper status"

This change could hide problems in jenkins configuration.

Tomas, opinion?
Comment 6 Mark Tompsett 2018-05-07 16:03:43 UTC
Created attachment 75136 [details] [review]
Bug 20505: Friendlier death message

Actually telling the user where to look to solve the problem
is more helpful than just expecting them to know why the selenium
driver didn't load.

Failures should be in the test file, not nested deep in subcalls.

prove t/db_dependent/selenium/
Comment 7 David Cook 2022-12-06 05:25:12 UTC
Closing this due to age and since koha-testing-docker makes this much easier these days