Description
Alex Buckley
2017-07-23 11:01:24 UTC
Created attachment 65193 [details] [review] Selenium test for logging into the Koha intranet and OPAC` Created attachment 65194 [details] [review] Bug 18974 - Selenium test for searching for an item in the intranet and opac Created attachment 65265 [details] [review] Bug 18974 - Selenium test for creating a currency, budget and fund Created attachment 65522 [details] [review] Bug 18974 - Selenium test for purchase workflow This test covers the creation of purchase suggestion in the OPAC, creation of vendor, basket and the creation of the order from the suggested item Created attachment 65867 [details] [review] Bug 18974 - Selenium test for purchase workflow This test covers the creation of purchase suggestion in the OPAC. The approval of that purchase suggestion in the intranet and then the creation of vendor, basket and the creation of the order from the suggested item Test plan: 1. Make sure you have a superlibrarian patron with the name koha and password koha 2. Make sure you have an active budget and fund both with a value of greater than 0 3. Make sure you set the staffClientBaseURL and OPACBaseURL to their appropriate values for your environment. If you are testing using Kohadevbox then the corrext values for these sysprefs is localhost:8080 and http://localhost:80 respectively 4. Ensure you have your selenium service running 5. Navigate to t/db_dependent/selenium 6. Enter Koha shell: sudo koha-shell <instancename> 7. perl purachse_workflow.t 8. The test takes a while to excute but notice that all tests pass Created attachment 66154 [details] [review] Bug 18974 - Added selenium test for the Koha web installer and onboarding tool Test plan: 1. wget https://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.0.jar 2.vim /etc/apt/sources.list.d/firefox.list 3. Paste in: deb http://packages.linuxmint.com debian import 4. suod apt-get update 5. sudo apt-get install firefox 6. sudo apt-get install xvfb 7. SELENIUM_PATH=/home/vagrant/kohaclone/selenium-server-standalone-2.53.0.jar 8. Xvfb :1 -screen 0 1024x768x24 2>&1 >/dev/null & 9. DISPLAY=:1 java -jar $SELENIUM_PATH 10. In a new terminal restart memcached and drop and recreate the Koha database 11. sudo koha-shell kohadev 12. perl t/db_dependent/selenium/installkoha.t 13. Notice the db errors saying that database tables do not exist 14. Notice the CP test output messages stating what the test is doing Created attachment 66163 [details] [review] Bug 18974 - Selenium test for cataloguing MARC21 records and adding items Created attachment 66509 [details] [review] Bug 18974 - Selenium test for cataloguing MARC21 records and adding items This Selenium test performs the cataloging workflow starting on the intranets home page and then using z39.50 targets to add a MARC record. After that items are added to the MARC record Test plan (this test plan includes how to install Selenium as well as how to run this test, for the benefit of people who have not got Selenium installed on their machines): 1. wget https://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.0.jar 2. vim /etc/apt/sources.list.d/firefox.list 3. Paste into the file: deb http://packages.linuxmint.com debian import 4. sudo apt-get update 5. sudo apt-get install firefox 6. sudo apt-get install xvfb 7. Set the SELENIUM_PATH vartiable: SELENIUM_PATH=/home/vagrant/kohaclone/selenium-server-standalone-2.53.0.jar 8. Xvfb :1 -screen 0 1024x768x24 2>&1 >/dev/null & 9. DISPLAY=:1 java -jar $SELENIUM_PATH Note: This will start up the selenium server. Everytime you want to shutdown this terminal window and want to restart Selenium just run the step 9 command to restart the Selenium server 10. Open a new terminal window and write in: git clone https://github.com/gempesaw/Selenium-Remote-Driver --branch build/master --single-branch --depth 1 11. cd Selenium-Remote-Driver 12. perl Makefile.PL 13. make 14. make test 15. sudo make install 16. Now everything is installed and you can et up for running the selenium tests 17. Drop and recreate you Koha database and restart memcached 18. Go through the web installer (installing sample z39.50 targets) and the onboarding tool 19. Create a superlibrarian user with the username koha and password koha 20. If your usual port configuration for the intranet and OPAC is 8081 and 8080 respectively then set the staffClientBaseURL and OPACBaseURL system preferences to localhost:8080 and localhost:80 respectively 21. sudo koha-shell <instancename> 22. perl t/db_dependent/selenium/cataloging_workflow.t 23. The test should pass. Note: The time_diff comments in the test output showing what the test is doing Note: If you have issues with installing Selenium and Selenium::Remote::Driver please write a comment on the bug report and I will be more than happy to create a screencapture video showing all the steps Sponsored-By: Catalyst IT Created attachment 68816 [details] [review] Bug 18974 - Selenium test for cataloguing MARC21 records and adding items This Selenium test performs the cataloging workflow starting on the intranets home page and then using z39.50 targets to add a MARC record. After that items are added to the MARC record Test plan (this test plan includes how to install Selenium as well as how to run this test, for the benefit of people who have not got Selenium installed on their machines): 1. wget https://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.0.jar 2. vim /etc/apt/sources.list.d/firefox.list 3. Paste into the file: deb http://packages.linuxmint.com debian import 4. sudo apt-get update 5. sudo apt-get install firefox 6. sudo apt-get install xvfb 7. Set the SELENIUM_PATH vartiable: SELENIUM_PATH=/home/vagrant/kohaclone/selenium-server-standalone-2.53.0.jar 8. Xvfb :1 -screen 0 1024x768x24 2>&1 >/dev/null & 9. DISPLAY=:1 java -jar $SELENIUM_PATH Note: This will start up the selenium server. Everytime you want to shutdown this terminal window and want to restart Selenium just run the step 9 command to restart the Selenium server 10. Open a new terminal window and write in: git clone https://github.com/gempesaw/Selenium-Remote-Driver --branch build/master --single-branch --depth 1 11. cd Selenium-Remote-Driver 12. perl Makefile.PL 13. make 14. make test 15. sudo make install 16. Now everything is installed and you can et up for running the selenium tests 17. Drop and recreate you Koha database and restart memcached 18. Go through the web installer (installing sample z39.50 targets) and the onboarding tool 19. Create a superlibrarian user with the username koha and password koha 20. If your usual port configuration for the intranet and OPAC is 8081 and 8080 respectively then set the staffClientBaseURL and OPACBaseURL system preferences to localhost:8080 and localhost:80 respectively 21. sudo koha-shell <instancename> 22. perl t/db_dependent/selenium/cataloging_workflow.t 23. The test should pass. Note: The time_diff comments in the test output showing what the test is doing Note: If you have issues with installing Selenium and Selenium::Remote::Driver please write a comment on the bug report and I will be more than happy to create a screencapture video showing all the steps Sponsored-By: Catalyst IT Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Created attachment 68817 [details] [review] Bug 18974: Change to use ENV variables and fix login bug The kohadevbox has admin/admin set, so for testing when creating the initial superuser, if you wish to minimize effort, use admin/admin. Otherwise, make sure to: export KOHA_USER={whatever your user is} export KOHA_PASS={whatever your password is} before attempting to run the test. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> The maintainability of this code is really difficult, because the xpath's used are cryptic. I would recommend adding less cryptic xpath's (id based) and/or comments describing what is being clicked or filled in. I'll let QA'rs decide if it is Failed QA for this. Blocked by bug 19243. We need to find a clean way to continue. As I don't have time to work on this at present I am unassigning myself in case someone else wants to take it on. The tests need to be rewritten completely. V1 of rewrite: diff: https://gitlab.com/tuxayo/Koha/-/commit/6f2aaf0867b3157d4823fe6c7e5449b49fe1133a whole test file: https://gitlab.com/tuxayo/Koha/-/blob/6f2aaf0867b3157d4823fe6c7e5449b49fe1133a/t/db_dependent/selenium/cataloging_workflow.t There are currently 2 big holes in my rewrite. They are the TODOs in the code, I'm copying them here. # TODO test fails depending on what we have from LOC. # e.g. If Title or Transcribing agency is not provided # * tag 040 subfield c Transcribing agency in tab 0 # * tag 245 subfield a Title in tab 2 # * Field 040 is mandatory, at least one of its subfields must be filled. # * Field 245 is mandatory, at least one of its subfields must be filled. # Idea: a simple & reliable way to handle this would be to unconditionally # fill 040$c and 245. # Before doing that (inventing my thing): has this problem already been solved in our knowledge in a better way? # TODO we have the issue of when we run the test twice, the record exists # and Koha asks about what to do about this. # Is there an obvious way to cleanup this? # notes: the need is to cleanup after the test sucess # and to cleanup before starting the test again in the case after a failure. # Will bug 19821 help? # Bug 19821 - Run tests on a separate database # Idea from the current cleanup(): (whose code seems mostly not related to this test) # - name the record specifically like test_cataloging_workflow # - delete it and it's related items # - question: isn't it risky about data integrity? # Another idea: # - pick a random number and replace the ISBN by it (In reply to Victor Grousset/tuxayo from comment #16) > There are currently 2 big holes in my rewrite. They are the TODOs in the > code, I'm copying them here. > > # TODO test fails depending on what we have from LOC. > # e.g. If Title or Transcribing agency is not provided > # * tag 040 subfield c Transcribing agency in tab 0 > # * tag 245 subfield a Title in tab 2 > # * Field 040 is mandatory, at least one of its subfields must be filled. > # * Field 245 is mandatory, at least one of its subfields must be filled. > > # Idea: a simple & reliable way to handle this would be to unconditionally > # fill 040$c and 245. > # Before doing that (inventing my thing): has this problem already been > solved in our knowledge in a better way? I am not sure I understand. Make the tests pass using the default framework will be a good first step. > # TODO we have the issue of when we run the test twice, the record exists > # and Koha asks about what to do about this. > # Is there an obvious way to cleanup this? > # notes: the need is to cleanup after the test sucess > # and to cleanup before starting the test again in the case after a failure. > # Will bug 19821 help? > # Bug 19821 - Run tests on a separate database See what is done in other selenium tests, using cleanup() > # Idea from the current cleanup(): (whose code seems mostly not related to > this test) > # - name the record specifically like test_cataloging_workflow > # - delete it and it's related items > # - question: isn't it risky about data integrity? Data integrity? Why? > # Another idea: > # - pick a random number and replace the ISBN by it You can generate random data using TestBuilder is you like, then delete the object and reuse the data, like: my $patron = $builder->build_object({ class => 'Koha::Patrons' }); my $patron_info = $patron->unblessed; $patron->delete; # reuse $patron_info, you may need to remove some key, like IDs In basic_workflow.t we use hardcoded values, that can work too. (In reply to Jonathan Druart from comment #17) > I am not sure I understand. > Make the tests pass using the default framework will be a good first step. The issue is that I wasn't getting the same record from LOC every time. And some have missing 040 and 245. Which doesn't comply with the default framework. I've fixed this by searching for an ISBN instead of a title + author. > tuxayo: > > # Idea from the current cleanup(): (whose code seems mostly not related to > > this test) > > # - name the record specifically like test_cataloging_workflow > > # - delete it and it's related items > > # - question: isn't it risky about data integrity? > > Data integrity? Why? Due to direct SQL DB modification. Actually it's not risky, that just means I need advice from someone who knows what do delete to cleanup a record + item creation. i.e. In which tables they span + eventual stuff that the Koha data layer does. > > # Another idea: > > # - pick a random number and replace the ISBN by it > > You can generate random data using TestBuilder is you like, then delete the > object and reuse the data, like: Thanks, that will be handy for other tests. For this one, the need is more low level right? (A random ISBN to replace the one that came from Z39.50 to avoid collisions). So, easy also. > In basic_workflow.t we use hardcoded values, that can work too. It won't help here because the issue is when the test is ran twice. But before going into this (cleaning up) or creating unique data that won't collide went test is ran twice: Should we rely on bug 19821 instead? That would block this patch and there other (less than a month, on maybe I'm too optimistic) but allow to not add and test code to handle cleanup. Including the case of the test that failed or was interrupted. We aim for the new tests to be backportable right? If so, do you think bug 19821 could be backported? I can spend time on that for each branch (especially since I would like to improve the coverage down to my branch ^^) but not matter how much, I don't know if it would be OK from a QA POV. If we aim to backport and depending on bug 19821 prevents that, then I would lean towards doing the current way of cleanup. And benefit from bug 19821 only in a future cycle. == Annex == Current state: diff: https://gitlab.com/tuxayo/Koha/-/commit/8935d1edc50156cea9dc16b56284c2994d941a0d whole test file: https://gitlab.com/tuxayo/Koha/-/blob/8935d1edc50156cea9dc16b56284c2994d941a0d/t/db_dependent/selenium/cataloging_workflow.t (In reply to Victor Grousset/tuxayo from comment #18) > (In reply to Jonathan Druart from comment #17) > > I am not sure I understand. > > Make the tests pass using the default framework will be a good first step. > > The issue is that I wasn't getting the same record from LOC every time. And > some have missing 040 and 245. Which doesn't comply with the default > framework. > > I've fixed this by searching for an ISBN instead of a title + author. In a first step I'd not rely on network (so no search from LOC). > > tuxayo: > > > # Idea from the current cleanup(): (whose code seems mostly not related to > > > this test) > > > # - name the record specifically like test_cataloging_workflow > > > # - delete it and it's related items > > > # - question: isn't it risky about data integrity? > > > > Data integrity? Why? > > Due to direct SQL DB modification. Actually it's not risky, that just means > I need advice from someone who knows what do delete to cleanup a record + > item creation. i.e. In which tables they span + eventual stuff that the Koha > data layer does. I'd delete the item and record using selenium (click delete buttons). Then you need: itemtype and branches. Actually will you know which data you need to delete as you will have to create them. > > > # Another idea: > > > # - pick a random number and replace the ISBN by it > > > > You can generate random data using TestBuilder is you like, then delete the > > object and reuse the data, like: > > Thanks, that will be handy for other tests. For this one, the need is more > low level right? (A random ISBN to replace the one that came from Z39.50 to > avoid collisions). So, easy also. > > > In basic_workflow.t we use hardcoded values, that can work too. > > It won't help here because the issue is when the test is ran twice. It should not, if you cleanup if won't face issues if the test is ran several times. > But before going into this (cleaning up) or creating unique data that won't > collide went test is ran twice: Should we rely on bug 19821 instead? > That would block this patch and there other (less than a month, on maybe I'm > too optimistic) but allow to not add and test code to handle cleanup. > Including the case of the test that failed or was interrupted. No, don't rely on bug 19821. You will have to cleanup anyway. If the test explodes in mid-air there will be data that won't be removed. Don't bother about that, it is how it works already. > We aim for the new tests to be backportable right? > If so, do you think bug 19821 could be backported? I can spend time on that > for each branch (especially since I would like to improve the coverage down > to my branch ^^) but not matter how much, I don't know if it would be OK > from a QA POV. Master first, usually enhancements are not backported. (In reply to Jonathan Druart from comment #19) > In a first step I'd not rely on network (so no search from LOC). Good idea. > I'd delete the item and record using selenium (click delete buttons). Thanks, this is perfect to test these things also :) > Then you need: itemtype and branches. Actually will you know which data you > need to delete as you will have to create them. I don't get this. Is there anything to cleanup other than the record and item created just before? > No, don't rely on bug 19821. You will have to cleanup anyway. Ok, no bug 19821 > If the test explodes in mid-air there will be data that won't be removed. > Don't bother about that, it is how it works already. - if the test explodes before any creation: delete nothing - if the test explodes after record creation: delete record - if the test explodes after item creation or succeeds: delete booth - if the deletion test fails: ??? Is there a less complex way to handle this? Like having a test that creates, deletes and that's all. In case of a mid-air failure, no separate cleanup. How should it impact the other tests? If they rely on a precise number of order of the items and records, then isn't that the issue? Like that means we can't at all change the initial test data. > > We aim for the new tests to be backportable right? > > If so, do you think bug 19821 could be backported? I can spend time on that > > for each branch (especially since I would like to improve the coverage down > > to my branch ^^) but not matter how much, I don't know if it would be OK > > from a QA POV. > > Master first, usually enhancements are not backported. That should be worth it if we are lucky that it works immediately which is very likely in stable in beginning of the cycle. Updated draft. - no use of Z39.05, cataloging form instead - deleting of item and record at in the last steps of the test https://gitlab.com/tuxayo/Koha/-/blob/1e68bf89da3ddc919b2a395bf6cb1bc3c7ba93d0/t/db_dependent/selenium/cataloging_workflow.t Created attachment 111191 [details] [review] Bug 18974 - Selenium test for cataloguing MARC21 records and adding items This Selenium test performs the cataloging workflow starting on the intranets home page and then using z39.50 targets to add a MARC record. After that items are added to the MARC record Test plan (this test plan includes how to install Selenium as well as how to run this test, for the benefit of people who have not got Selenium installed on their machines): 1. wget https://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.0.jar 2. vim /etc/apt/sources.list.d/firefox.list 3. Paste into the file: deb http://packages.linuxmint.com debian import 4. sudo apt-get update 5. sudo apt-get install firefox 6. sudo apt-get install xvfb 7. Set the SELENIUM_PATH vartiable: SELENIUM_PATH=/home/vagrant/kohaclone/selenium-server-standalone-2.53.0.jar 8. Xvfb :1 -screen 0 1024x768x24 2>&1 >/dev/null & 9. DISPLAY=:1 java -jar $SELENIUM_PATH Note: This will start up the selenium server. Everytime you want to shutdown this terminal window and want to restart Selenium just run the step 9 command to restart the Selenium server 10. Open a new terminal window and write in: git clone https://github.com/gempesaw/Selenium-Remote-Driver --branch build/master --single-branch --depth 1 11. cd Selenium-Remote-Driver 12. perl Makefile.PL 13. make 14. make test 15. sudo make install 16. Now everything is installed and you can et up for running the selenium tests 17. Drop and recreate you Koha database and restart memcached 18. Go through the web installer (installing sample z39.50 targets) and the onboarding tool 19. Create a superlibrarian user with the username koha and password koha 20. If your usual port configuration for the intranet and OPAC is 8081 and 8080 respectively then set the staffClientBaseURL and OPACBaseURL system preferences to localhost:8080 and localhost:80 respectively 21. sudo koha-shell <instancename> 22. perl t/db_dependent/selenium/cataloging_workflow.t 23. The test should pass. Note: The time_diff comments in the test output showing what the test is doing Note: If you have issues with installing Selenium and Selenium::Remote::Driver please write a comment on the bug report and I will be more than happy to create a screencapture video showing all the steps Sponsored-By: Catalyst IT Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Created attachment 111192 [details] [review] Bug 18974: Selenium test for cataloguing MARC21 records and adding items This Selenium test performs the cataloging workflow starting on the intranets home page and then using z39.50 targets to add a MARC record. After that items are added to the MARC record Test plan (this test plan includes how to install Selenium as well as how to run this test, for the benefit of people who have not got Selenium installed on their machines): 1. wget https://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.0.jar 2. vim /etc/apt/sources.list.d/firefox.list 3. Paste into the file: deb http://packages.linuxmint.com debian import 4. sudo apt-get update 5. sudo apt-get install firefox 6. sudo apt-get install xvfb 7. Set the SELENIUM_PATH vartiable: SELENIUM_PATH=/home/vagrant/kohaclone/selenium-server-standalone-2.53.0.jar 8. Xvfb :1 -screen 0 1024x768x24 2>&1 >/dev/null & 9. DISPLAY=:1 java -jar $SELENIUM_PATH Note: This will start up the selenium server. Everytime you want to shutdown this terminal window and want to restart Selenium just run the step 9 command to restart the Selenium server 10. Open a new terminal window and write in: git clone https://github.com/gempesaw/Selenium-Remote-Driver --branch build/master --single-branch --depth 1 11. cd Selenium-Remote-Driver 12. perl Makefile.PL 13. make 14. make test 15. sudo make install 16. Now everything is installed and you can et up for running the selenium tests 17. Drop and recreate you Koha database and restart memcached 18. Go through the web installer (installing sample z39.50 targets) and the onboarding tool 19. Create a superlibrarian user with the username koha and password koha 20. If your usual port configuration for the intranet and OPAC is 8081 and 8080 respectively then set the staffClientBaseURL and OPACBaseURL system preferences to localhost:8080 and localhost:80 respectively 21. sudo koha-shell <instancename> 22. perl t/db_dependent/selenium/cataloging_workflow.t 23. The test should pass. Note: The time_diff comments in the test output showing what the test is doing Note: If you have issues with installing Selenium and Selenium::Remote::Driver please write a comment on the bug report and I will be more than happy to create a screencapture video showing all the steps Sponsored-By: Catalyst IT Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Created attachment 111193 [details] [review] Bug 18974: Change to use ENV variables and fix login bug The kohadevbox has admin/admin set, so for testing when creating the initial superuser, if you wish to minimize effort, use admin/admin. Otherwise, make sure to: export KOHA_USER={whatever your user is} export KOHA_PASS={whatever your password is} before attempting to run the test. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Created attachment 111194 [details] [review] Bug 18974: Rewrite test using latest conventions + change strategy I mean conventions from current tests, administration_tasks.t was used as a model. Changes in the test logic: - don't use Z39.50, use cataloging form - test item and record deletion Created attachment 111195 [details] [review] Bug 18974: Rewrite test using latest conventions + change strategy i.e. conventions from current tests, administration_tasks.t was used as a model. Changes in the test logic: - don't use Z39.50, use cataloging form - test item and record deletion == Test plan == 1. Run the test 2. It should pass! Rebased Fixed commit message format Simplified selectors Rather than go through the setup process detailed in the patches, with KTD you can start koha-testing-docker with: ktd --selenium up I think that should take care of all the setup requirements, but I could be wrong. Hopefully ktd --selenium up should take care of the requirements now. This worked with bug 30645 Revised test plan: 1. Start up koha-testing-docker (KTD) with selenium: ktd --selenium up 2. Apply the patch. 3. Run the tests: prove t/db_dependent/selenium/cataloging_workflow.t 4. The tests should pass! Unsurprisingly (since this one has been around for a while), the new test fails for me: prove t/db_dependent/selenium/cataloging_workflow.t t/db_dependent/selenium/cataloging_workflow.t .. 1/1 STRACE: /usr/share/perl5/Try/Tiny.pm:123 in Selenium::Remote::Driver::catch {...} /usr/share/perl5/Selenium/Remote/Driver.pm:361 in Try::Tiny::try (eval 1729):1 in Selenium::Remote::Driver::__ANON__ (eval 1731):2 in Selenium::Remote::Driver::__ANON__ /usr/share/perl5/Selenium/Remote/Driver.pm:1340 in Selenium::Remote::Driver::_execute_command /usr/share/perl5/Selenium/Remote/Driver.pm:1340 in (eval) /kohadevbox/koha/t/lib/Selenium.pm:174 in Selenium::Remote::Driver::find_element t/db_dependent/selenium/cataloging_workflow.t:62 in t::lib::Selenium::click Error while executing command: no such element: Unable to locate element: //div[@id="container-main"]//a[contains(@href, "/cataloguing/addbooks.pl")] at /usr/share/perl5/Selenium/Remote/Driver.pm line 411. at /usr/share/perl5/Selenium/Remote/Driver.pm line 356. # Looks like your test exited with 255 just after 1. t/db_dependent/selenium/cataloging_workflow.t .. Dubious, test returned 255 (wstat 65280, 0xff00) All 1 subtests passed Test Summary Report ------------------- t/db_dependent/selenium/cataloging_workflow.t (Wstat: 65280 Tests: 1 Failed: 0) Non-zero exit status: 255 Files=1, Tests=1, 13 wallclock secs ( 0.01 usr 0.00 sys + 0.66 cusr 0.10 csys = 0.77 CPU) Result: FAIL |