Bug 19185

Summary: Web installer and onboarding tool selenium test
Product: Koha Reporter: Alex Buckley <alexbuckley>
Component: Test SuiteAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Tomás Cohen Arazi <tomascohen>
Severity: enhancement    
Priority: P5 - low CC: jonathan.druart, katrin.fischer, kyle, martin.renvoize, mtompset, tomascohen, victor
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18974
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21149
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17360
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
21.11.00
Bug Depends on: 19243, 19821    
Bug Blocks: 19384, 29485    
Attachments: Bug 19185 - Selenium test for the Koha web installer and onboarding tool
Bug 19185: Change to use ENV, fix login and passwords
Bug 19185: Use submit button if only one exists on the page
Bug 19185: Ease elements selection
Bug 19185: Add selenium tests for installation and onboarding process
Bug 19185: Use submit button if only one exists on the page
Bug 19185: Ease elements selection
Bug 19185: Add selenium tests for installation and onboarding process
Bug 19185: Use submit button if only one exists on the page
Bug 19185: Ease elements selection
Bug 19185: Add selenium tests for installation and onboarding process
Bug 19185: Add selenium tests for installation and onboarding process
Bug 19185: Use submit button if only one exists on the page
Bug 19185: Use submit button if only one exists on the page
Bug 19185: Ease elements selection
Bug 19185: Add selenium tests for installation and onboarding process
Bug 19185: Fix tests
Bug 19185: Use submit button if only one exists on the page
Bug 19185: Ease elements selection
Bug 19185: Add selenium tests for installation and onboarding process
Bug 19185: Fix tests
Bug 19185: Fix language selection
Bug 19185: DB must exist but empty
Bug 19185: Split installation and onboarding
Bug 19185: Add missing +x on new test files
Bug 19185: Fix regressions.t
Bug 19185: Fix regressions.t
Bug 19185: Make sure user is logged in before adding records to the cart

Description Alex Buckley 2017-08-27 04:18:12 UTC
This Selenium test will go through the workflow for installing Koha including going through the web installer and onboarding tool
Comment 1 Alex Buckley 2017-08-27 04:26:19 UTC
Created attachment 66508 [details] [review]
Bug 19185 - Selenium test for the Koha web installer and onboarding tool

This Selenium test will go through the web installer and onboarding tool
testing the Koha installation workflow.

Test plan (this test plan includes how to install Selenium as well as how to run this test, for the benefit of people who have not got
Selenium installed on their machines):

1. wget https://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.0.jar

2. vim /etc/apt/sources.list.d/firefox.list

3. Paste into the file:
deb http://packages.linuxmint.com debian import

4. sudo apt-get update

5. sudo apt-get install firefox

6. sudo apt-get install xvfb

7. Set the SELENIUM_PATH vartiable:
SELENIUM_PATH=/home/vagrant/kohaclone/selenium-server-standalone-2.53.0.jar

8. Xvfb :1 -screen 0 1024x768x24 2>&1 >/dev/null &

9. DISPLAY=:1 java -jar $SELENIUM_PATH

Note: This will start up the selenium server. Everytime you want to
shutdown this terminal window and want to restart Selenium just run the step 9 command to restart the Selenium server

10. Open a new terminal window and write in:
git clone https://github.com/gempesaw/Selenium-Remote-Driver --branch build/master --single-branch --depth 1

11. cd Selenium-Remote-Driver

12. perl Makefile.PL

13. make

14. make test

15. sudo make install

16. Now everything is installed and you can et up for running the
selenium tests

17. In a new terminal restart memcached and drop and recreate the Koha database

18. sudo koha-shell kohadev

19. perl t/db_dependent/selenium/installkoha.t

20. Notice the db errors saying that database tables do not exist

21. Notice the CP test output messages stating what the test is doing

Note: warns exist in this selenium test because using Selenium to test
the web installer and onboarding tool has proved to be flaky. The use of warning out the title of the currently loaded page and implementing pauses has helped get the test to pass after many hours of troubleshooting. If anyone can suggest alternatives please do so.

Sponsored-By: Catalyst IT
Comment 2 Alex Buckley 2017-08-27 04:27:09 UTC
This patch was originally attached to 18974 however I have placed it in its own bug report to make testing more straightforward
Comment 3 Mark Tompsett 2017-10-30 04:27:02 UTC
Created attachment 68830 [details] [review]
Bug 19185: Change to use ENV, fix login and passwords

The kohadevbox has admin/admin set, so for testing when
creating the initial superuser, if you wish to minimize
effort, use admin/admin. Otherwise, make sure to:
export KOHA_USER={whatever your user is}
export KOHA_PASS={whatever your password is}
before attempting to run the test.

Also, this test was failing because 'sub auth' was using
the ugliest of xpath's to find the submit button.
I've noticed the other selenium tests have similar functions.
It might be an idea to build a t/lib/Selenium.pm so as to
reduce code duplication and older versions differing from
newer versions.
Comment 4 Mark Tompsett 2017-10-30 04:27:54 UTC
Even with my attached patch, I hit problems.
Comment 5 Jonathan Druart 2017-10-30 13:36:24 UTC
Blocked by bug 19243. We need to find a clean way to continue.
Comment 6 Jonathan Druart 2018-05-04 14:31:19 UTC
I am working on this.
Comment 7 Jonathan Druart 2018-05-04 17:53:43 UTC
Created attachment 75088 [details] [review]
Bug 19185: Use submit button if only one exists on the page

During the installation process we do not have fieldset.action, in order
to not complicate changes we are just going to use the submit button if
only one exists.
Comment 8 Jonathan Druart 2018-05-04 17:53:47 UTC
Created attachment 75089 [details] [review]
Bug 19185: Ease elements selection

- The marcflavour select needs an id to be selected easily
- "Continue to the next step" should be a button (like others)
Comment 9 Jonathan Druart 2018-05-04 17:53:50 UTC
Created attachment 75090 [details] [review]
Bug 19185: Add selenium tests for installation and onboarding process
Comment 10 Jonathan Druart 2018-05-04 17:55:40 UTC
Maybe not ready yet for integration but need feedbacks.
Comment 11 Martin Renvoize 2018-08-03 21:30:43 UTC
Comment on attachment 75090 [details] [review]
Bug 19185: Add selenium tests for installation and onboarding process

Review of attachment 75090 [details] [review]:
-----------------------------------------------------------------

::: t/db_dependent/selenium/00-installation.t
@@ +52,5 @@
> +    my $db_pass = C4::Context->config('pass');
> +
> +    $driver->get($base_url."mainpage.pl");
> +
> +    my $lang = "en"; # The idea here is to loop on all languages

Question: Why would we want to loop for all languages? Are you still intending on implementing this?
Comment 12 Martin Renvoize 2018-08-03 21:46:36 UTC
This generally looks pretty solid to me.. certainly a good first step.. i'm sure we could test the negative cases a bit more (for example using bad db credentials and checking the right errors are present), but generally speaking this looks good to me.  Of course, these tests can only be run on a brand new DB and as such make most sense when used alongside the dependency bug 19821 which given the right config will create a brand new clean database for each test run.
Comment 13 Jonathan Druart 2018-08-07 13:42:28 UTC
(In reply to Martin Renvoize from comment #11)
> Comment on attachment 75090 [details] [review] [review]
> Bug 19185: Add selenium tests for installation and onboarding process
> 
> Review of attachment 75090 [details] [review] [review]:
> -----------------------------------------------------------------
> 
> ::: t/db_dependent/selenium/00-installation.t
> @@ +52,5 @@
> > +    my $db_pass = C4::Context->config('pass');
> > +
> > +    $driver->get($base_url."mainpage.pl");
> > +
> > +    my $lang = "en"; # The idea here is to loop on all languages
> 
> Question: Why would we want to loop for all languages? Are you still
> intending on implementing this?

We should test the installer process for different languages, in the script I have listed the ones we have for the manual. The idea would be to run it for each of them.
We should also test with and without sample data (i.e. without and with onboarding tool), see commented tests.

But it's for the next steps (if it does not take too long to be part of the codebase).
Comment 14 Martin Renvoize 2018-11-02 16:27:23 UTC
Created attachment 81913 [details] [review]
Bug 19185: Use submit button if only one exists on the page

During the installation process we do not have fieldset.action, in order
to not complicate changes we are just going to use the submit button if
only one exists.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 15 Martin Renvoize 2018-11-02 16:27:26 UTC
Created attachment 81914 [details] [review]
Bug 19185: Ease elements selection

- The marcflavour select needs an id to be selected easily
- "Continue to the next step" should be a button (like others)

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 16 Martin Renvoize 2018-11-02 16:27:29 UTC
Created attachment 81915 [details] [review]
Bug 19185: Add selenium tests for installation and onboarding process

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 17 Martin Renvoize 2018-11-02 16:27:56 UTC
With a bit of perseverance, I got it to work. Signing Off
Comment 18 Martin Renvoize 2018-11-02 16:28:54 UTC
As CI maintainer, I've set Tomas to QA this one.. I think he's best placed to give it the final go ahead.
Comment 19 Katrin Fischer 2019-02-15 07:11:49 UTC
BLOCKED by FQA on the dependent bug for the moment.
Comment 20 Jonathan Druart 2019-02-17 13:52:40 UTC
Back to SO.
Comment 21 Katrin Fischer 2019-05-11 19:23:37 UTC
I've applied both bugs, bug 19821 and this one.

With Joubu's help (thx!) my Selenium tests run in general now, but these sadly still fail:

t/db_dependent/selenium/00-installation.t .. 
STRACE:	/usr/share/perl5/Try/Tiny.pm:124 in Selenium::Remote::Driver::catch {...} 
	/usr/local/share/perl/5.24.1/Selenium/Remote/Driver.pm:348 in Try::Tiny::try
	(eval 330):1 in Selenium::Remote::Driver::__ANON__
	(eval 332):2 in Selenium::Remote::Driver::__ANON__
	/usr/local/share/perl/5.24.1/Selenium/Remote/Driver.pm:1230 in Selenium::Remote::Driver::_execute_command
	/usr/local/share/perl/5.24.1/Selenium/Remote/Driver.pm:1230 in (eval)
	/home/vagrant/kohaclone/t/lib/Selenium.pm:120 in Selenium::Remote::Driver::find_element
	t/db_dependent/selenium/00-installation.t:63 in t::lib::Selenium::fill_form

sh: 1: curl: not found
malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/share/perl5/JSON.pm line 171.
# Looks like your test exited with 127 before it could output anything.
t/db_dependent/selenium/00-installation.t .. Dubious, test returned 127 (wstat 32512, 0x7f00)
Failed 2/2 subtests 

Test Summary Report
-------------------
t/db_dependent/selenium/00-installation.t (Wstat: 32512 Tests: 0 Failed: 0)
  Non-zero exit status: 127
  Parse errors: Bad plan.  You planned 2 tests but ran 0.
Comment 22 Katrin Fischer 2019-05-11 19:27:33 UTC
Hm, looks like curl was missing, installed and tried again.

It looks like it failed at logging in, did I miss a preparation step?

kohadev-koha@kohadevbox:/home/vagrant/kohaclone$ prove t/db_dependent/selenium/00-installation.t
t/db_dependent/selenium/00-installation.t .. 
STRACE:	/usr/share/perl5/Try/Tiny.pm:124 in Selenium::Remote::Driver::catch {...} 
	/usr/local/share/perl/5.24.1/Selenium/Remote/Driver.pm:348 in Try::Tiny::try
	(eval 330):1 in Selenium::Remote::Driver::__ANON__
	(eval 332):2 in Selenium::Remote::Driver::__ANON__
	/usr/local/share/perl/5.24.1/Selenium/Remote/Driver.pm:1230 in Selenium::Remote::Driver::_execute_command
	/usr/local/share/perl/5.24.1/Selenium/Remote/Driver.pm:1230 in (eval)
	/home/vagrant/kohaclone/t/lib/Selenium.pm:120 in Selenium::Remote::Driver::find_element
	t/db_dependent/selenium/00-installation.t:63 in t::lib::Selenium::fill_form


SCREENSHOT: https://framapic.org/lOB1mxx6MxFE/TSvv7dPkJkCy
An element could not be located on the page using the given search parameters: //*[@id="language"],xpath at /home/vagrant/kohaclone/t/lib/Selenium.pm line 120.
# Looks like your test exited with 255 before it could output anything.
t/db_dependent/selenium/00-installation.t .. Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 2/2 subtests 

Test Summary Report
-------------------
t/db_dependent/selenium/00-installation.t (Wstat: 65280 Tests: 0 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 2 tests but ran 0.
Comment 23 Jonathan Druart 2019-05-12 17:24:10 UTC
Test plan (for bug 19185 and bug 19821):
% prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
Config entry 'database_test' does not exist at /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 18

# Edit $KOHA_CONF, add a database_test entry identical to database (then restart_all):
% prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
Entries 'database_test' and 'database' have the same value in your config at /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 20.

# Edit $KOHA_CONF, edit database_test with koha_kohadev_test (then restart_all)
% prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
Access denied for user 'koha_kohadev'@'localhost' to database 'koha_kohadev_test'

# Login as the admin mysql user
MariaDB [(none)]> GRANT ALL PRIVILEGES ON `koha_kohadev_test`.* TO 'koha_kohadev'@'localhost';

% prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
t/db_dependent/selenium/00-installation.t .. ok 
All tests successful.
Files=1, Tests=2, 25 wallclock secs ( 0.02 usr  0.01 sys +  1.34 cusr  0.24 csys =  1.61 CPU)
Result: PASS
Comment 24 Katrin Fischer 2019-10-08 11:47:21 UTC
We've discussed this and the other patch set on bug 19812 during the hackfest and decided to wait until Jonathan's return to pick it up again. IIRC we were not sure if bug 19821 is really needed for this one to proceed and how they fit together. There was also a question on permissions etc. for the second database needed and how this would work for the various installation scenarios we currently support.

Jonathan, could you get in touch with me once around again? Maybe also something we could clear up in a dev meeting.
Comment 25 Jonathan Druart 2019-10-15 09:02:26 UTC
(In reply to Katrin Fischer from comment #24)
> We've discussed this and the other patch set on bug 19812 during the
> hackfest and decided to wait until Jonathan's return to pick it up again.
> IIRC we were not sure if bug 19821 is really needed for this one to proceed
> and how they fit together. There was also a question on permissions etc. for
> the second database needed and how this would work for the various
> installation scenarios we currently support.
> 
> Jonathan, could you get in touch with me once around again? Maybe also
> something we could clear up in a dev meeting.

Hi Katrin,
We definitely need bug 19821 to make those tests work correctly. Did you write the questions down?
Comment 26 Jonathan Druart 2020-01-11 15:54:52 UTC
*** Bug 17360 has been marked as a duplicate of this bug. ***
Comment 27 Victor Grousset/tuxayo 2020-04-21 03:37:09 UTC
Tried to test this as related to bug 19821

What does this mean?↓

> # Edit $KOHA_CONF, add a database_test entry identical to database (then restart_all):
Comment 28 Jonathan Druart 2020-04-21 08:47:43 UTC
(In reply to Victor Grousset/tuxayo from comment #27)
> Tried to test this as related to bug 19821
> 
> What does this mean?↓
> 
> > # Edit $KOHA_CONF, add a database_test entry identical to database (then restart_all):

database and database_test with the same value, like:
  <database>koha_kohadev</database>
  <database_test>koha_kohadev</database_test>
Comment 29 Victor Grousset/tuxayo 2020-04-21 19:53:12 UTC
Created attachment 103406 [details] [review]
Bug 19185: Use submit button if only one exists on the page

During the installation process we do not have fieldset.action, in order
to not complicate changes we are just going to use the submit button if
only one exists.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Comment 30 Victor Grousset/tuxayo 2020-04-21 19:53:16 UTC
Created attachment 103407 [details] [review]
Bug 19185: Ease elements selection

- The marcflavour select needs an id to be selected easily
- "Continue to the next step" should be a button (like others)

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Comment 31 Victor Grousset/tuxayo 2020-04-21 19:53:19 UTC
Created attachment 103408 [details] [review]
Bug 19185: Add selenium tests for installation and onboarding process

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

== Test plan ==
1. Apply the patches (including bug 19821)
2. restart_all
3. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
4. result: Config entry 'database_test' does not exist at
     /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 18
5. Edit $KOHA_CONF, add a database_test entry identical to database
     You should have:
     <database>koha_kohadev</database>
     <database_test>koha_kohadev</database_test>
6. restart_all
7. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
8. result: Entries 'database_test' and 'database' have the same value in your
     config at /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 20.
9. Edit $KOHA_CONF, edit database_test with koha_test as DB name
10. restart_all
11. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
12. result: Access denied for user 'koha_kohadev'@'localhost'
       to database 'koha_test'
13. Connect to the DBMS
14. Example for koha_testing_docker
      docker exec -it koha_db_1 bash
      mysql -u root --password=password
15. GRANT ALL PRIVILEGES ON `koha_test`.* TO 'koha_kohadev';
16. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
17. result: All tests successful.
Comment 32 Victor Grousset/tuxayo 2020-04-21 19:56:03 UTC
Created attachment 103416 [details] [review]
Bug 19185: Add selenium tests for installation and onboarding process

== Test plan ==
1. Apply the patches (including bug 19821)
2. restart_all
3. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
4. result: Config entry 'database_test' does not exist at
     /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 18
5. Edit $KOHA_CONF, add a database_test entry identical to database
     You should have:
     <database>koha_kohadev</database>
     <database_test>koha_kohadev</database_test>
6. restart_all
7. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
8. result: Entries 'database_test' and 'database' have the same value in your
     config at /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 20.
9. Edit $KOHA_CONF, edit database_test with koha_test as DB name
10. restart_all
11. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
12. result: Access denied for user 'koha_kohadev'@'localhost'
       to database 'koha_test'
13. Connect to the DBMS
14. Example for koha_testing_docker
      docker exec -it koha_db_1 bash
      mysql -u root --password=password
15. GRANT ALL PRIVILEGES ON `koha_test`.* TO 'koha_kohadev';
16. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
17. result: All tests successful.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Comment 33 Victor Grousset/tuxayo 2020-04-21 19:59:13 UTC
Signed off, I did nothing more than what's the test plan in the patch so I hope I didn't missed something when I rewrote it.

== Diff of the test plan ==
=== Before ===
The test plan was in comment 23

% prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
Config entry 'database_test' does not exist at /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 18

# Edit $KOHA_CONF, add a database_test entry identical to database (then restart_all):
% prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
Entries 'database_test' and 'database' have the same value in your config at /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 20.

# Edit $KOHA_CONF, edit database_test with koha_kohadev_test (then restart_all)
% prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
Access denied for user 'koha_kohadev'@'localhost' to database 'koha_kohadev_test'

# Login as the admin mysql user
MariaDB [(none)]> GRANT ALL PRIVILEGES ON `koha_kohadev_test`.* TO 'koha_kohadev'@'localhost';

% prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
t/db_dependent/selenium/00-installation.t .. ok 
All tests successful.
Files=1, Tests=2, 25 wallclock secs ( 0.02 usr  0.01 sys +  1.34 cusr  0.24 csys =  1.61 CPU)
Result: PASS


=== After ===
1. Apply the patches (including bug 19821)
2. restart_all
3. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
4. result: Config entry 'database_test' does not exist at
     /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 18
5. Edit $KOHA_CONF, add a database_test entry identical to database
     You should have:
     <database>koha_kohadev</database>
     <database_test>koha_kohadev</database_test>
6. restart_all
7. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
8. result: Entries 'database_test' and 'database' have the same value in your
     config at /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 20.
9. Edit $KOHA_CONF, edit database_test with koha_test as DB name
10. restart_all
11. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
12. result: Access denied for user 'koha_kohadev'@'localhost'
       to database 'koha_test'
13. Connect to the DBMS
14. Example for koha_testing_docker
      docker exec -it koha_db_1 bash
      mysql -u root --password=password
15. GRANT ALL PRIVILEGES ON `koha_test`.* TO 'koha_kohadev';
16. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
17. result: All tests successful.
Comment 34 Kyle M Hall 2020-06-12 13:35:04 UTC
Created attachment 105796 [details] [review]
Bug 19185: Use submit button if only one exists on the page

During the installation process we do not have fieldset.action, in order
to not complicate changes we are just going to use the submit button if
only one exists.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 35 Kyle M Hall 2020-06-12 13:35:26 UTC
Created attachment 105797 [details] [review]
Bug 19185: Use submit button if only one exists on the page

During the installation process we do not have fieldset.action, in order
to not complicate changes we are just going to use the submit button if
only one exists.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 36 Kyle M Hall 2020-06-12 13:36:14 UTC
Created attachment 105798 [details] [review]
Bug 19185: Ease elements selection

- The marcflavour select needs an id to be selected easily
- "Continue to the next step" should be a button (like others)

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 37 Kyle M Hall 2020-06-12 13:36:18 UTC
Created attachment 105799 [details] [review]
Bug 19185: Add selenium tests for installation and onboarding process

== Test plan ==
1. Apply the patches (including bug 19821)
2. restart_all
3. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
4. result: Config entry 'database_test' does not exist at
     /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 18
5. Edit $KOHA_CONF, add a database_test entry identical to database
     You should have:
     <database>koha_kohadev</database>
     <database_test>koha_kohadev</database_test>
6. restart_all
7. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
8. result: Entries 'database_test' and 'database' have the same value in your
     config at /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 20.
9. Edit $KOHA_CONF, edit database_test with koha_test as DB name
10. restart_all
11. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
12. result: Access denied for user 'koha_kohadev'@'localhost'
       to database 'koha_test'
13. Connect to the DBMS
14. Example for koha_testing_docker
      docker exec -it koha_db_1 bash
      mysql -u root --password=password
15. GRANT ALL PRIVILEGES ON `koha_test`.* TO 'koha_kohadev';
16. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
17. result: All tests successful.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 38 Jonathan Druart 2020-06-15 08:43:05 UTC
This depends on bug 19821 that is not PQA yet.
Comment 39 Jonathan Druart 2021-04-28 14:36:41 UTC
Created attachment 120273 [details] [review]
Bug 19185: Fix tests
Comment 40 Victor Grousset/tuxayo 2021-05-03 22:32:28 UTC
Still works :)
Comment 41 Jonathan Druart 2021-10-21 08:48:57 UTC
Created attachment 126646 [details] [review]
Bug 19185: Use submit button if only one exists on the page

During the installation process we do not have fieldset.action, in order
to not complicate changes we are just going to use the submit button if
only one exists.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 42 Jonathan Druart 2021-10-21 08:49:02 UTC
Created attachment 126647 [details] [review]
Bug 19185: Ease elements selection

- The marcflavour select needs an id to be selected easily
- "Continue to the next step" should be a button (like others)

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 43 Jonathan Druart 2021-10-21 08:49:07 UTC
Created attachment 126648 [details] [review]
Bug 19185: Add selenium tests for installation and onboarding process

== Test plan ==
1. Apply the patches (including bug 19821)
2. restart_all
3. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
4. result: Config entry 'database_test' does not exist at
     /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 18
5. Edit $KOHA_CONF, add a database_test entry identical to database
     You should have:
     <database>koha_kohadev</database>
     <database_test>koha_kohadev</database_test>
6. restart_all
7. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
8. result: Entries 'database_test' and 'database' have the same value in your
     config at /home/vagrant/kohaclone/t/lib/Bootstrap.pm line 20.
9. Edit $KOHA_CONF, edit database_test with koha_test as DB name
10. restart_all
11. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
12. result: Access denied for user 'koha_kohadev'@'localhost'
       to database 'koha_test'
13. Connect to the DBMS
14. Example for koha_testing_docker
      docker exec -it koha_db_1 bash
      mysql -u root --password=password
15. GRANT ALL PRIVILEGES ON `koha_test`.* TO 'koha_kohadev';
16. prove -Mt::lib::Bootstrap t/db_dependent/selenium/00-installation.t
17. result: All tests successful.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 44 Jonathan Druart 2021-10-21 08:49:12 UTC
Created attachment 126649 [details] [review]
Bug 19185: Fix tests
Comment 45 Jonathan Druart 2021-10-21 08:49:16 UTC
Created attachment 126650 [details] [review]
Bug 19185: Fix language selection

'en' is in an optgroup
Comment 46 Jonathan Druart 2021-10-21 08:49:21 UTC
Created attachment 126651 [details] [review]
Bug 19185: DB must exist but empty
Comment 47 Jonathan Druart 2021-10-21 08:49:25 UTC
Created attachment 126652 [details] [review]
Bug 19185: Split installation and onboarding

The previous installation tests were testing the full onboarding process
by not selecting all the sample data.
It would be good to test the whole sample data set.

So there are now 2 files, 00-onboarding.t and 01-installation.t
Comment 48 Jonathan Druart 2021-10-25 09:44:31 UTC
Pushed to master for 21.11, thanks to everybody involved!
Comment 49 Jonathan Druart 2021-10-25 11:09:56 UTC
Created attachment 126819 [details] [review]
Bug 19185: Add missing +x on new test files
Comment 50 Victor Grousset/tuxayo 2021-10-25 15:55:28 UTC
I see that it's ran only by the CI:
>    skip "This test must be run with an empty DB. We are using KOHA_TESTING that is set by our CI\nIf you really want to run it, set this env var.", 2 unless $ENV{KOHA_TESTING};

Shouldn't the output of skip end up in the terminal when running prove?
Tests finished very (1sec) fast on local Koha and only reading the code allowed me to know the cause.
Linked to this is the fact that we use skip to check for Selenium lib being installed. So based on what I see, it should skip (an mark as passed!) the tests when Selenium lib isn't there :o


What would be the simplest way to run the full test suite locally?

> prove -r t/ xt/ -j $(nproc) --rules='seq=t/db_dependent/**.t'
> echo "DROP DATABASE koha_kohadev; CREATE DATABASE koha_kohadev;" | koha-mysql kohadev
> restart_all
> prove t/db_dependent/selenium/
Comment 51 Jonathan Druart 2021-10-26 10:24:09 UTC
Created attachment 126901 [details] [review]
Bug 19185: Fix regressions.t

The following test is failing:
| #   Failed test 'OPAC - Remove from cart'
| #   at t/db_dependent/selenium/regressions.t line 132.
| Can't call method "get_value" on an undefined value at t/db_dependent/selenium/regressions.t line 110.
| # Looks like your test exited with 2 just after 3.
| [12:14:08] t/db_dependent/selenium/regressions.t

We are dependind on the search engine and the records in the DB but the
installer is not inserting any records.

This patch is suggesting to reuse the code from search_utf8
(and so make it reusable first) for remove_from_cart test.
This code is mocking the Zebra index with some MARC data and so the
search will return results. We will finally be able to click on the
add to/remove from cart links.
Comment 52 Jonathan Druart 2021-10-26 10:26:43 UTC
(In reply to Victor Grousset/tuxayo from comment #50)

My first goal is to make Jenkins happy, then we will see what needs to be improved and how.

To see how Jenkins run the full test suite, have a look at https://gitlab.com/koha-community/koha-testing-docker/-/blob/master/files/run.sh (last else block for the non-light builds).
Comment 53 Tomás Cohen Arazi 2021-10-26 14:36:39 UTC
Created attachment 126918 [details] [review]
Bug 19185: Fix regressions.t

The following test is failing:
| #   Failed test 'OPAC - Remove from cart'
| #   at t/db_dependent/selenium/regressions.t line 132.
| Can't call method "get_value" on an undefined value at t/db_dependent/selenium/regressions.t line 110.
| # Looks like your test exited with 2 just after 3.
| [12:14:08] t/db_dependent/selenium/regressions.t

We are dependind on the search engine and the records in the DB but the
installer is not inserting any records.

This patch is suggesting to reuse the code from search_utf8
(and so make it reusable first) for remove_from_cart test.
This code is mocking the Zebra index with some MARC data and so the
search will return results. We will finally be able to click on the
add to/remove from cart links.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 54 Tomás Cohen Arazi 2021-10-26 14:38:19 UTC
Good job, Jonathan.

Old tests still pass with the refactoring, new tests now pass with and empty DB, which is what KTD will do in run.sh.
Comment 55 Jonathan Druart 2021-10-26 14:43:36 UTC
Follow-up pushed to master.
Comment 56 Jonathan Druart 2021-10-27 12:32:37 UTC
Created attachment 126984 [details] [review]
Bug 19185: Make sure user is logged in before adding records to the cart