During the hackfest 15 in Marseille, akafred present us how he integrated acceptance tests in Koha using Cucumber. I would like to get developer's opinions on the need of this kind of tests into Koha.
Created attachment 36940 [details] [review] Bug 13849: POC - my first Cucumber contact During the hackfest 15 in Marseille, akafred present us how he integrated acceptance tests in Koha using Cucumber. With this patch, I would like to get other developer opinions on the usefulness of this kind of tests for the Koha software. To test: 0/ * Install the selenium server (have a look at bug 13691) * Install libtest-bdd-cucumber-perl (pkg) and App::pherkin (cpan) * Edit t/db_dependent/cucumber/features/step_definitions/basic_steps.pl and fill the login and password of a superlibrarian user. Maybe you should also change the address of the selenium server (search for 'remote_server_addr'). 1/ Start the selenium server and launch the tests: $ cd t/db_dependent/cucumber $ pherkin or $ prove tests.t Even if the second method is recommended in Test::BDD::Cucumber::Manual::Integration, I prefer the first one Indeed, the tests will pass if a step is not defined (maybe something is wrong somewhere...).
Hi there! Awesome that somebody else is in flames about Cucumber! I have been writing the Cucumber tests into another directory structure, and I have encountered some issues with it. My file hierarchy: kohaclone/t/Cucumber/runCucumberTests.pl #Binds *::Cucumber::Manual::Integration #And runs all Cucumber tests. kohaclone/t/Cucumber/Overdues/features/*.feature #Feature definitions kohaclone/t/Cucumber/Overdues/features/step_definitions/*_steps.pl #Step handlers kohaclone/t/Cucumber/TestObjects/Koha/Overdues/*.pm #Packages to actually #implement the test code. I cannot set debugger breakpoints #into the step-files, because of how the Test::BDD-modules #load them as code closures, so I implement the business logic #in the TestObjects. Return the comparable values and run the #Test::More function on the returned values, eg two objects or #an error code. Note regarding passing test data arrays in the Scenario description: Examples: | branchCode | borrowerCategory | letterNumber | | | S | 1 | | | S | 2 | | | S | 3 | | | PT | 1 | | | PT | 2 | | | PT | 3 | | CPL | PT | 1 | | CPL | PT | 2 | | FTL | YA | 1 | | FTL | YA | 2 | | FTL | YA | 3 | Instead of getting the array values from the step-definition regexp, you can get them from the StepContext-object: See my attachment scenarioNumberForTableDriveScenarios.txt
Created attachment 36942 [details] scenarioNumberForTableDrivenScenarios. Keep track of the Scenario data array location to get data as HASH instead of ARRAY.
Created attachment 37622 [details] Example Cucumber tests for the overdues rewrite Having written a new overdue gathering and sending module and using Cucumber tests, here are the test features I used.
Created attachment 37678 [details] Trigger script for Cucumber tests suite
Some tutorial on how to do Cukes in Koha! http://pastebin.com/3Tya8kwR
To test: 0/ * Install the selenium server (have a look at bug 13691) * Install libtest-bdd-cucumber-perl (pkg) and App::pherkin (cpan) * Edit t/db_dependent/cucumber/features/step_definitions/basic_steps.pl and fill the login and password of a superlibrarian user. Maybe you should also change the address of the selenium server (search for 'remote_server_addr'). 1/ Start the selenium server and launch the tests: $ cd t/db_dependent/cucumber $ pherkin or $ prove tests.t Hi Jonathan, I tried your patch using pherkin and prove, and I have the same issue at both times : Create a Category is OK [...] But create a patron crash after submit : Create a patron As a librarian I want to create a new patron Scenario: Create a patron Given I am logged in as a library admin Given I have gone to the new patron page When I have filled the field surname with "test_patron_surname" And I have filled the field cardnumber with "4242424242" And I have filled the field userid with "test_username" And I have filled the field password with "password" And I have filled the field password2 with "password" And I have clicked on the submit button Then The patron "test_username" should have been created step defined at features/1-patron.feature line 22. ok 1 - Starting to execute step: The patron "test_username" should have been created not ok 2 - The patron test_username should exist # Failed test 'The patron test_username should exist' # at features/step_definitions/1-patron_steps.pl line 12. # got: '0' # expected: '1' 1..2
All works perfectly here. Make sure you have a superlibrarian user "koha" with pwd "koha" or edit features/step_definitions/basic_steps.pl
Works fine now My problem was the 'BorrowerMandatoryField' syspref which was waiting more fields filled than tests did.
Created attachment 39983 [details] [review] Bug 13849 : Adding Cucumber tests to create a new biblio To test: 0/ * Install the selenium server (have a look at bug 13691) * Install libtest-bdd-cucumber-perl (pkg) and App::pherkin (cpan) * Edit t/db_dependent/cucumber/features/step_definitions/basic_steps.pl and fill the login and password of a superlibrarian user. Maybe you should also change the address of the selenium server (search for 'remote_server_addr'). 1/ Start the selenium server and launch the tests: $ cd t/db_dependent/cucumber $ pherkin or $ prove tests.t This patch add a new scenario creating a new biblio
Created attachment 40070 [details] Cucumber test suite Vaara-style I discussed with Oslo about how to best organize these Cucumber tests. They hinted that we should use Page object pattern to do these Cucumber/Selenium tests. Which is a jolly good idea! I haven't had time to do any of those Page Object Pattern-driven objects for hiding ugly implementation specifics for web-crawling. I also have some new features in production utilizing Cucumber tests and I started to encounter some test structuring issues. So I cooked up a better way to organize the test components. See testsStructuring.tutorial in the .tar.gz This is 100% compatible with the runCucumberTests.t-script and IMHO makes for a much more beautiful code structuring than the way of doing things in the Perl documentation for Cucumber.
I don't see any obvious advantage to including Cucumber tests but happy to be convinced otherwise. Is the idea that the test reports are prettier and the scenarios more verbose?
It's no longer relevant (to me). The idea was to let anybody write tests, even without technical skills. Another goal was automatically generate (internationalized) screenshots for the manual https://lists.katipo.co.nz/pipermail/koha/2018-January/049807.html
Abandoned. See bug 13849.
Abandoned. See bug 34076.