Bug 26162 - Prevent Selenium's StaleElementReferenceException
Summary: Prevent Selenium's StaleElementReferenceException
Status: Pushed to oldoldstable
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low normal (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Keywords:
Depends on: 26143
Blocks:
  Show dependency treegraph
 
Reported: 2020-08-06 13:08 UTC by Jonathan Druart
Modified: 2020-09-20 12:51 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:
20.11.00, 20.05.03, 19.11.09, 19.05.15


Attachments
Bug 26162: Make Selenium click action more robust (1.44 KB, patch)
2020-08-06 13:11 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 26162: Make Selenium click action more robust (2.99 KB, patch)
2020-08-06 13:11 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 26162: Make Selenium click action more robust (3.05 KB, patch)
2020-08-07 07:32 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 26162: Wait for the table to be refreshed (3.80 KB, patch)
2020-08-10 10:29 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 26162: Don't fall into an infinite loop (1.36 KB, patch)
2020-08-19 05:27 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 26162: [19.11] Fix DT info label (1.39 KB, patch)
2020-08-19 05:27 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2020-08-06 13:08:26 UTC
Got something new on Master_U18 #862 after bug 26143.


18:47:07 selenium_1   | 09:47:07.478 WARN - Exception: Element not found in the cache - perhaps the page has changed since it was looked up
18:47:07 selenium_1   | For documentation on this error, please visit: http://seleniumhq.org/exceptions/stale_element_reference.html
18:47:07 selenium_1   | Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
18:47:07 selenium_1   | System info: host: '78b9a07f51f2', ip: '192.168.16.2', os.name: 'Linux', os.arch: 'amd64', os.version: '4.19.0-9-amd64', java.version: '1.8.0_91'
18:47:07 selenium_1   | Driver info: driver.version: unknown
18:47:07 koha_1       | 
18:47:07 koha_1       | STRACE:	/usr/share/perl5/Try/Tiny.pm:123 in Selenium::Remote::Driver::catch {...} 
18:47:07 koha_1       | 	/usr/local/share/perl/5.26.1/Selenium/Remote/Driver.pm:353 in Try::Tiny::try
18:47:07 koha_1       | 	(eval 1571):1 in Selenium::Remote::Driver::__ANON__
18:47:07 koha_1       | 	(eval 1573):2 in Selenium::Remote::Driver::__ANON__
18:47:07 koha_1       | 	(eval 1546):17 in Selenium::Remote::Driver::_execute_command
18:47:07 koha_1       | 	/usr/local/share/perl/5.26.1/Selenium/Remote/WebElement.pm:63 in Selenium::Remote::WebElement::_execute_command
18:47:07 koha_1       | 	/kohadevbox/koha/t/lib/Selenium.pm:184 in Selenium::Remote::WebElement::click
18:47:07 koha_1       | 	/kohadevbox/koha/t/lib/Selenium.pm:172 in t::lib::Selenium::click_when_visible
18:47:07 koha_1       | 	t/db_dependent/selenium/administration_tasks.t:131 in t::lib::Selenium::click


It's failing consistently on D11 locally.
Comment 2 Jonathan Druart 2020-08-06 13:11:54 UTC
Created attachment 107889 [details] [review]
Bug 26162: Make Selenium click action more robust

See
https://stackoverflow.com/questions/12967541/how-to-avoid-staleelementreferenceexception-in-selenium
https://www.selenium.dev/exceptions/
https://developer.mozilla.org/en-US/docs/Web/WebDriver/Errors/StaleElementReference

This patch will fix the following failure we get under D11:
18:47:07 selenium_1   | 09:47:07.478 WARN - Exception: Element not found in the cache - perhaps the page has changed since it was looked up
18:47:07 selenium_1   | For documentation on this error, please visit: http://seleniumhq.org/exceptions/stale_element_reference.html
18:47:07 selenium_1   | Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
18:47:07 selenium_1   | System info: host: '78b9a07f51f2', ip: '192.168.16.2', os.name: 'Linux', os.arch: 'amd64', os.version: '4.19.0-9-amd64', java.version: '1.8.0_91'
18:47:07 selenium_1   | Driver info: driver.version: unknown
18:47:07 koha_1       |
18:47:07 koha_1       | STRACE:	/usr/share/perl5/Try/Tiny.pm:123 in Selenium::Remote::Driver::catch {...}
18:47:07 koha_1       | 	/usr/local/share/perl/5.26.1/Selenium/Remote/Driver.pm:353 in Try::Tiny::try
18:47:07 koha_1       | 	(eval 1571):1 in Selenium::Remote::Driver::__ANON__
18:47:07 koha_1       | 	(eval 1573):2 in Selenium::Remote::Driver::__ANON__
18:47:07 koha_1       | 	(eval 1546):17 in Selenium::Remote::Driver::_execute_command
18:47:07 koha_1       | 	/usr/local/share/perl/5.26.1/Selenium/Remote/WebElement.pm:63 in Selenium::Remote::WebElement::_execute_command
18:47:07 koha_1       | 	/kohadevbox/koha/t/lib/Selenium.pm:184 in Selenium::Remote::WebElement::click
18:47:07 koha_1       | 	/kohadevbox/koha/t/lib/Selenium.pm:172 in t::lib::Selenium::click_when_visible
18:47:07 koha_1       | 	t/db_dependent/selenium/administration_tasks.t:131 in t::lib::Selenium::click
Comment 3 Martin Renvoize 2020-08-07 07:32:11 UTC
Created attachment 107940 [details] [review]
Bug 26162: Make Selenium click action more robust

See
https://stackoverflow.com/questions/12967541/how-to-avoid-staleelementreferenceexception-in-selenium
https://www.selenium.dev/exceptions/
https://developer.mozilla.org/en-US/docs/Web/WebDriver/Errors/StaleElementReference

This patch will fix the following failure we get under D11:
18:47:07 selenium_1   | 09:47:07.478 WARN - Exception: Element not found in the cache - perhaps the page has changed since it was looked up
18:47:07 selenium_1   | For documentation on this error, please visit: http://seleniumhq.org/exceptions/stale_element_reference.html
18:47:07 selenium_1   | Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
18:47:07 selenium_1   | System info: host: '78b9a07f51f2', ip: '192.168.16.2', os.name: 'Linux', os.arch: 'amd64', os.version: '4.19.0-9-amd64', java.version: '1.8.0_91'
18:47:07 selenium_1   | Driver info: driver.version: unknown
18:47:07 koha_1       |
18:47:07 koha_1       | STRACE:	/usr/share/perl5/Try/Tiny.pm:123 in Selenium::Remote::Driver::catch {...}
18:47:07 koha_1       | 	/usr/local/share/perl/5.26.1/Selenium/Remote/Driver.pm:353 in Try::Tiny::try
18:47:07 koha_1       | 	(eval 1571):1 in Selenium::Remote::Driver::__ANON__
18:47:07 koha_1       | 	(eval 1573):2 in Selenium::Remote::Driver::__ANON__
18:47:07 koha_1       | 	(eval 1546):17 in Selenium::Remote::Driver::_execute_command
18:47:07 koha_1       | 	/usr/local/share/perl/5.26.1/Selenium/Remote/WebElement.pm:63 in Selenium::Remote::WebElement::_execute_command
18:47:07 koha_1       | 	/kohadevbox/koha/t/lib/Selenium.pm:184 in Selenium::Remote::WebElement::click
18:47:07 koha_1       | 	/kohadevbox/koha/t/lib/Selenium.pm:172 in t::lib::Selenium::click_when_visible
18:47:07 koha_1       | 	t/db_dependent/selenium/administration_tasks.t:131 in t::lib::Selenium::click

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 4 Martin Renvoize 2020-08-07 07:33:06 UTC
The code makes sense and reads well.. QA script passes.

I tested this by running the relevant test in a loop over 50 iterations and didn't get the errors expecting in a D11 k-t-d.

Signing off.
Comment 5 Jonathan Druart 2020-08-07 14:56:23 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 6 Jonathan Druart 2020-08-10 10:29:48 UTC
Created attachment 107985 [details] [review]
Bug 26162: Wait for the table to be refreshed

The previous patch did not work as expected. We still got a
StaleElementReference exception.
But this time on
10:43:47 selenium_1   | Caused by: org.openqa.selenium.NoSuchElementException: Unable to locate element: {"method":"xpath","selector":"//*[@id=\"branchname\"]"}
Because we found the one that existed on the page, not the one sent back
in AJAX.

The idea of this patch is to search for the "Showing 1 to X of Y entries" info and wait for X == Y
Comment 7 Jonathan Druart 2020-08-10 10:32:57 UTC
Last patch pushed to master for 20.11
Comment 8 Lucas Gass 2020-08-13 18:31:45 UTC
backported to 20.05.x for 20.05.03
Comment 9 Aleisha Amohia 2020-08-16 22:00:39 UTC
backported to 19.11.x for 19.11.09
Comment 10 Jonathan Druart 2020-08-19 05:27:01 UTC
Created attachment 108568 [details] [review]
Bug 26162: Don't fall into an infinite loop

If something went wrong we do want to stop the script!
Comment 11 Jonathan Druart 2020-08-19 05:27:04 UTC
Created attachment 108569 [details] [review]
Bug 26162: [19.11] Fix DT info label

There is an inconsistency in 19.11:
% grep Showing koha-tmpl/intranet-tmpl/prog/en/includes/datatables.inc koha-tmpl/intranet-tmpl/prog/js/datatables.js
koha-tmpl/intranet-tmpl/prog/en/includes/datatables.inc:    var MSG_DT_INFO = _("Showing _START_ to _END_ of _TOTAL_");
koha-tmpl/intranet-tmpl/prog/js/datatables.js:        "sInfo"             : window.MSG_DT_INFO || "Showing _START_ to _END_ of _TOTAL_ entries",

The "entries" is not always displayed.

datatables.inc has been removed from 20.05 by
Bug 24662: Remove global variables MSG_* from datatables.inc
Comment 12 Jonathan Druart 2020-08-19 05:28:41 UTC
We had administration_tasks.t in infinite loop on all 19.11 jobs.

The first patch is for all branches and will prevent this problem to appear in the future.

The second patch is for <=19.11 only
Comment 13 Aleisha Amohia 2020-08-19 05:59:04 UTC
last two patches backported to 19.11.x for 19.11.09
Comment 14 Jonathan Druart 2020-08-19 07:37:49 UTC
Patch "Bug 26162: Don't fall into an infinite loop"

Pushed to master for 20.11
Comment 15 Jonathan Druart 2020-09-03 14:20:08 UTC
So, this approach did not fix the problem.

We got another failure: https://jenkins.koha-community.org/job/Koha_Master_U18/886/consoleFull

Screenshot is: https://snipboard.io/nxYFhq.jpg

15:20:58 selenium_1   | 06:20:58.918 INFO - Executing: [click: 35 [[FirefoxDriver: firefox on LINUX (67a62258-11cc-423b-b6e7-0e9e8b66997a)] -> xpath: //div[@class="main container-fluid"]//a[contains(@href, "/admin/branches.pl?op=add_form&branchcode=UT_BC")]]])
15:21:19 selenium_1   | 06:21:19.815 WARN - Exception thrown
15:21:19 selenium_1   | org.openqa.selenium.StaleElementReferenceException: Element not found in the cache - perhaps the page has changed since it was looked up

15:21:40 koha_1       | STRACE:	/usr/share/perl5/Try/Tiny.pm:123 in Selenium::Remote::Driver::catch {...} 
15:21:40 koha_1       | 	/usr/local/share/perl/5.26.1/Selenium/Remote/Driver.pm:353 in Try::Tiny::try
15:21:40 koha_1       | 	(eval 1579):1 in Selenium::Remote::Driver::__ANON__
15:21:40 koha_1       | 	(eval 1581):2 in Selenium::Remote::Driver::__ANON__
15:21:40 koha_1       | 	/usr/local/share/perl/5.26.1/Selenium/Remote/Driver.pm:1330 in Selenium::Remote::Driver::_execute_command
15:21:40 koha_1       | 	/usr/local/share/perl/5.26.1/Selenium/Remote/Driver.pm:1330 in (eval)
15:21:40 koha_1       | 	/kohadevbox/koha/t/lib/Selenium.pm:126 in Selenium::Remote::Driver::find_element
15:21:40 koha_1       | 	t/db_dependent/selenium/administration_tasks.t:137 in t::lib::Selenium::fill_form



That's getting really (really!) annoying...
Comment 16 Jonathan Druart 2020-09-03 16:13:52 UTC
Master_D10 #342 (Docker_3)
Comment 17 Victor Grousset/tuxayo 2020-09-20 12:51:22 UTC
Backported to 19.05.x branch for 19.05.15