Bug 33370 - Cypress tests are failing randomly
Summary: Cypress tests are failing randomly
Status: RESOLVED DUPLICATE of bug 33408
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Keywords:
Depends on: 33400
Blocks: 33402
  Show dependency treegraph
 
Reported: 2023-03-30 12:31 UTC by Jonathan Druart
Modified: 2023-04-05 07:44 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 33370: Intercept and wait for avs and vendors GETs (1.22 KB, patch)
2023-03-30 14:41 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 33370: Create a new visit_and_wait command (12.75 KB, patch)
2023-04-04 08:09 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2023-03-30 12:31:30 UTC
On Koha_Master/2446
koha_1       |     <testcase name="Agreement CRUD operations Add agreement" time="0.0000" classname="Add agreement">
koha_1       |       <failure message="Timed out retrying after 4000ms: Expected to find element: `main div[class=&apos;dialog alert&apos;]`, but never found it." type="AssertionError"><![CDATA[AssertionError: Timed out retrying after 4000ms: Expected to find element: `main div[class='dialog alert']`, but never found it.
koha_1       |     at Context.eval (http://kohadev-intra.mydnsname.org:8081/__cypress/tests?p=t/cypress/integration/Agreements_spec.ts:352:12)]]></failure>
koha_1       |     </testcase>
Comment 1 Jonathan Druart 2023-03-30 12:32:23 UTC
I did recreate locally, using run_all_tests.pl (so from the CLI)

    <testcase name="Agreement CRUD operations Add agreement" time="0.0000" classname="Add agreement">     
      <failure message="Timed out retrying after 4000ms: Expected to find element: `main div[class=&apos;dialog alert&apos;]`, but never found it." type="AssertionError"><![CDATA[AssertionError: Timed out retrying after 4000ms: Expected to find element: `main div[class='dialog alert']`, but never found it.            
    at Context.eval (http://kohadev-intra.mydnsname.org:8081/__cypress/tests?p=t/cypress/integration/Agreements_spec.ts:352:12)]]></failure>                                                                         
    </testcase>
Comment 2 Jonathan Druart 2023-03-30 12:32:48 UTC
And I saw another one using Cypress UI

Timed out retrying after 4050ms: cy.click() failed because this element:

<a href="/cgi-bin/koha/erm/agreements/add" class="btn btn-default">...</a>

is being covered by another element:

<div class="modal_centered" data-v-018a4733="">...</div>

Fix this problem, or use {force: true} to disable error checking.Learn more
t/cypress/integration/Agreements_spec.ts:181:38
  179 |         // Click the button in the toolbar
  180 |         cy.visit("/cgi-bin/koha/erm/agreements");
> 181 |         cy.contains("New agreement").click();
Comment 3 Jonathan Druart 2023-03-30 12:42:40 UTC
This is what I am seeing: https://snipboard.io/Ts2hq8.jpg
Comment 4 Jonathan Druart 2023-03-30 12:53:19 UTC
Timed out retrying after 4050ms: cy.click() failed because this element:

<a href="/cgi-bin/koha/erm/agreements" class="">...</a>

is being covered by another element:

<div class="modal_centered" data-v-018a4733="">...</div>

Fix this problem, or use {force: true} to disable error checking.Learn more
t/cypress/integration/Agreements_spec.ts:141:55
  139 |         });
  140 |         cy.visit("/cgi-bin/koha/erm/erm.pl");
> 141 |         cy.get("#navmenulist").contains("Agreements").click();
      |                                                       ^
  142 |         cy.get("main div[class='dialog alert']").contains(
  143 |             /Something went wrong/
  144 |         );
Comment 5 Jonathan Druart 2023-03-30 12:54:03 UTC
It certainly happens because the main "loading" modal is on top of everything, we haven't retrieved the vendors and AVs and so the modal is still on top when we are trying to continue the tests.
Comment 6 Jonathan Druart 2023-03-30 14:41:45 UTC
Created attachment 149001 [details] [review]
Bug 33370: Intercept and wait for avs and vendors GETs
Comment 7 Jonathan Druart 2023-03-30 14:42:36 UTC
I think this patch is fixing the problem, but it feels weird to have to add the wait statement to after all our visit calls. Is there another better way?
Comment 8 Jonathan Druart 2023-03-31 11:19:54 UTC
Maybe we could overload cy.visit to not have to add the wait statement all over the tests (?)
Comment 9 Pedro Amorim 2023-04-01 15:28:01 UTC
This seems like a decent solution to the problem.

(In reply to Jonathan Druart from comment #8)
> Maybe we could overload cy.visit to not have to add the wait statement all
> over the tests (?)

What do you mean by overload cy.visit?
Comment 10 Jonathan Druart 2023-04-04 07:49:43 UTC
(In reply to Pedro Amorim from comment #9)
> This seems like a decent solution to the problem.
> 
> (In reply to Jonathan Druart from comment #8)
> > Maybe we could overload cy.visit to not have to add the wait statement all
> > over the tests (?)
> 
> What do you mean by overload cy.visit?

https://docs.cypress.io/api/cypress-api/custom-commands#Overwrite-Existing-Commands

But actually it might be better to create our own visit command instead.
Comment 11 Jonathan Druart 2023-04-04 08:09:56 UTC
Created attachment 149119 [details] [review]
Bug 33370: Create a new visit_and_wait command

It seems easier and cleaner to have our own visit_and_wait command that
will intercept the 2 queries (for avs and vendors) and then do the wait.

This is needed for the first hit to erm module.
Comment 12 Paul Derscheid 2023-04-04 08:12:47 UTC
This looks much cleaner. Good choice.
Comment 13 Jonathan Druart 2023-04-04 08:16:02 UTC
Note that the command is named "visit_and_wait", but it is actually ERM specific, it should be put in a "erm" subdirectory or renamed.
Comment 14 Jonathan Druart 2023-04-04 08:23:32 UTC
Titles is failing consistently with

    <testcase name="Title CRUD operations Add title" time="0.0000" classname="Add title">      <failure message="Timed out retrying after 4000ms: Expected to find element: `main div[class=&apos;dialog alert&apos;]`, but never found it." type="AssertionError"><![CDATA[AssertionError: Timed out retrying after 4000ms: Expected to find element: `main div[class='dialog alert']`, but never found it.

when run from the command line.
But passes when run from the Cypress UI. I think it has something to do with the size of the window (the alert appears at the top, and is not visible when we submit the form).
Comment 15 Jonathan Druart 2023-04-04 08:38:26 UTC
(In reply to Jonathan Druart from comment #14)
> Titles is failing consistently with
> 
>     <testcase name="Title CRUD operations Add title" time="0.0000"
> classname="Add title">      <failure message="Timed out retrying after
> 4000ms: Expected to find element: `main div[class=&apos;dialog
> alert&apos;]`, but never found it."
> type="AssertionError"><![CDATA[AssertionError: Timed out retrying after
> 4000ms: Expected to find element: `main div[class='dialog alert']`, but
> never found it.
> 
> when run from the command line.
> But passes when run from the Cypress UI. I think it has something to do with
> the size of the window (the alert appears at the top, and is not visible
> when we submit the form).

Moved to bug 33400.
Comment 16 Jonathan Druart 2023-04-05 05:53:53 UTC
Wondering if this actually needed if we have bug 33408, and specifically this change in Main.vue:

-    <div v-if="ERMModule">
+    <div v-if="initialized">
Comment 17 Jonathan Druart 2023-04-05 07:44:57 UTC
(In reply to Jonathan Druart from comment #16)
> Wondering if this actually needed if we have bug 33408, and specifically
> this change in Main.vue:
> 
> -    <div v-if="ERMModule">
> +    <div v-if="initialized">

Giving this one a try and closing this one for now.

*** This bug has been marked as a duplicate of bug 33408 ***