Bug 38236 - selenium/administration_tasks.t is failing randomly with "no such alert"
Summary: selenium/administration_tasks.t is failing randomly with "no such alert"
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low minor
Assignee: Chris Cormack
QA Contact: Testopia
URL:
Keywords:
Depends on: 25551
Blocks:
  Show dependency treegraph
 
Reported: 2024-10-23 11:31 UTC by Jonathan Druart
Modified: 2025-01-16 10:32 UTC (History)
1 user (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2024-10-23 11:31:13 UTC
15:44:00 koha_1       | STRACE:	/usr/share/perl5/Try/Tiny.pm:123 in Selenium::Remote::Driver::catch {...} 
15:44:00 koha_1       | 	/usr/share/perl5/Selenium/Remote/Driver.pm:361 in Try::Tiny::try
15:44:00 koha_1       | 	(eval 2020):1 in Selenium::Remote::Driver::__ANON__
15:44:00 koha_1       | 	(eval 2022):2 in Selenium::Remote::Driver::__ANON__
15:44:00 koha_1       | 	/usr/share/perl5/Selenium/Remote/Driver.pm:712 in Selenium::Remote::Driver::_execute_command
15:44:00 koha_1       | 	t/db_dependent/selenium/administration_tasks.t:180 in Selenium::Remote::Driver::accept_alert
15:44:00 koha_1       | 
15:44:00 selenium_1   | 1729518016380	Marionette	INFO	Stopped listening on port 46675
15:44:00 selenium_1   | JavaScript error: resource:///modules/Interactions.jsm, line 230: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
15:44:00 selenium_1   | 13:40:17.451 INFO [ActiveSessions$1.onStop] - Removing session 3ec3997e-064a-48e2-9ea2-f77f391fcf82 (org.openqa.selenium.firefox.GeckoDriverService)
15:44:00 koha_1       | Error while executing command: no such alert at /usr/share/perl5/Selenium/Remote/Driver.pm line 411.
15:44:00 koha_1       |  at /usr/share/perl5/Selenium/Remote/Driver.pm line 356.
15:44:00 koha_1       | # Looks like your test exited with 255 just after 3.
15:44:00 koha_1       | [13:40:17] t/db_dependent/selenium/administration_tasks.t ..............................
Comment 1 Jonathan Druart 2024-10-23 11:33:24 UTC
selenium/authentication_2fa.t had a similar failure (bug 32010)
Comment 2 Jonathan Druart 2024-10-23 11:33:46 UTC
It run 300x successfully locally.
Comment 3 Jonathan Druart 2024-10-23 11:34:50 UTC
It was Koha_Master/3063
Comment 4 Jonathan Druart 2024-10-23 11:37:20 UTC
(In reply to Jonathan Druart from comment #1)
> selenium/authentication_2fa.t had a similar failure (bug 32010)

But because of an AJAX request, what we do not have here.
Comment 5 Victor Grousset/tuxayo 2024-10-24 04:08:03 UTC
In case that helps, I'm able to reproduce this

```
	t/db_dependent/selenium/administration_tasks.t:179 in Selenium::Remote::Driver::find_element

Error while executing command: no such element: Unable to locate element: //input[@name="id"][@value=""]/following-sibling::button at /usr/share/perl5/Selenium/Remote/Driver.pm line 411.
```

It's the line just before the:
$s->driver->accept_alert; # Accept the modal "Are you sure you want to delete this authorized value?"
Comment 6 Jonathan Druart 2024-10-24 07:47:11 UTC
(In reply to Victor Grousset/tuxayo from comment #5)
> In case that helps, I'm able to reproduce this

How? Can you provide the screenshot of the problematic screen ?
Comment 7 Victor Grousset/tuxayo 2024-10-24 18:01:08 UTC
> How?

I'm just running the test in a loop.
3 such failures in 48 runs.

> Can you provide the screenshot of the problematic screen ?

Screenshot capture is broken in some failures. I had a patch to fix it that I added to a selenium ticket 6mo/1y ago. But it was mixed with other changes to testing process t/lib/Selenium.pm and ultimately wasn't retained. I'll try to find it.
Comment 8 Jonathan Druart 2024-10-28 10:58:05 UTC
Koha_Main/3075
Comment 9 Jonathan Druart 2024-10-28 11:00:18 UTC
(In reply to Victor Grousset/tuxayo from comment #7)
> > How?
> 
> I'm just running the test in a loop.
> 3 such failures in 48 runs.

I got 300 successes in a row :-/

> > Can you provide the screenshot of the problematic screen ?
> 
> Screenshot capture is broken in some failures. I had a patch to fix it that
> I added to a selenium ticket 6mo/1y ago. But it was mixed with other changes
> to testing process t/lib/Selenium.pm and ultimately wasn't retained. I'll
> try to find it.

I don't understand. Could you add a capture_screenshot call before the failure and see how is the screen?

  $driver->capture_screenshot( 'test.png' );
Comment 10 Victor Grousset/tuxayo 2024-12-04 05:41:59 UTC
(In reply to Jonathan Druart from comment #9)
> (In reply to Victor Grousset/tuxayo from comment #7)
> > Screenshot capture is broken in some failures. I had a patch to fix it that
> > I added to a selenium ticket 6mo/1y ago. But it was mixed with other changes
> > to testing process t/lib/Selenium.pm and ultimately wasn't retained. I'll
> > try to find it.
> 
> I don't understand. Could you add a capture_screenshot call before the
> failure and see how is the screen?
> 
>   $driver->capture_screenshot( 'test.png' );

Oh right! I though it was about the screenshot taken on failure.


So, after 6 failures in 188 runs

fail: https://lutim.lagout.org/8oEY7SVs/K9Vz9RK2.png

ok: https://lutim.lagout.org/gZ6IQwAu/f9MjqAoL.png

And turns out we are in a case where selenium_failure.png is managed to be taken.
And it the same image as a success!

It seems that the list of AVs hasn't loaded yet when it fails. And when selenium_failure.png is taken, the page has finished loading. Which is weird because Selenium is supposed to wait for the load event to be fired/document.readyState===complete.
And find_element() is supposed to wait until a timeout before failing. Given that the runtime of the failed run is 10 sec more than a good run, it seem it waited. But then somehow it didn't find the element, failed and the screenshot just after shows a loaded page.
Random Selenium bug? >_<

Anyway, does the screenshots give any hint?


code change to take screenshot:
> @@ -176,6 +176,7 @@ SKIP: {
> [...]
> #$driver->find_element('//input[@id="'.$av_id.'"]/following-sibling::button[text() = >"Delete"]')->click;
>+$driver->capture_screenshot( 'test.png' );
> $driver->find_element('//input[@name="id"][@value="'.$av_id.'"]/following-sibling::button')->click;
> $s->driver->accept_alert; # Accept the modal "Are you sure you want to delete this authorized value?"
Comment 11 Jonathan Druart 2024-12-04 08:56:19 UTC
It will take less time to rewrite those tests using cypress than debug them...
Comment 12 Jonathan Druart 2024-12-04 08:58:10 UTC
Could you remove this code and see if it still fails?


433             $("#Aform").submit(function() {
434                 if ( $('#authorised_value').length ) {
435                     if ( ! $('#authorised_value').get(0).checkValidity() ) {
436                         alert( _("Authorised value should be numeric.") );
437                         $('#authorised_value').focus();
438                         return false;
439                     }
440                 }
441                 return true;
442             });
Comment 13 Victor Grousset/tuxayo 2024-12-04 20:09:45 UTC
(In reply to Jonathan Druart from comment #11)
> It will take less time to rewrite those tests using cypress than debug
> them...

+1, as soon as bug 38503 makes it, I'll try to be able spend labor time rewriting tests.
Oh, it's in need signoff! I'll look at it ASAP! *jumping from excitement*

> Could you remove this code and see if it still fails?

4 fail in 155 runs
🫠🫠🫠
Comment 14 Jonathan Druart 2024-12-05 06:41:02 UTC
(In reply to Victor Grousset/tuxayo from comment #13)
> (In reply to Jonathan Druart from comment #11)
> > It will take less time to rewrite those tests using cypress than debug
> > them...
> 
> +1, as soon as bug 38503 makes it, I'll try to be able spend labor time
> rewriting tests.

Bug 38503 won't help much here.
Comment 15 Victor Grousset/tuxayo 2024-12-06 05:32:43 UTC
Yep, looking at the diff there, I just finally realized that mock means mock. And not touching the DB. 😅😅😅😅🤦
Anyway it's still a big win to be able to more easily test the front end.

> won't help much here

It might be coping and delusion but [to be continued in bug 29285 comment 4]
Comment 16 Jonathan Druart 2025-01-16 10:32:38 UTC
Koha_Main/3153 on Docker_15