I did recreate locally, using run_all_tests.pl (so from the CLI)

    <testcase name="Agreement CRUD operations Add agreement" time="0.0000" classname="Add agreement">     
      <failure message="Timed out retrying after 4000ms: Expected to find element: `main div[class=&apos;dialog alert&apos;]`, but never found it." type="AssertionError"><![CDATA[AssertionError: Timed out retrying after 4000ms: Expected to find element: `main div[class='dialog alert']`, but never found it.            
    at Context.eval (http://kohadev-intra.mydnsname.org:8081/__cypress/tests?p=t/cypress/integration/Agreements_spec.ts:352:12)]]></failure>                                                                         
    </testcase>

And I saw another one using Cypress UI

Timed out retrying after 4050ms: cy.click() failed because this element:

<a href="/cgi-bin/koha/erm/agreements/add" class="btn btn-default">...</a>

is being covered by another element:

<div class="modal_centered" data-v-018a4733="">...</div>

Fix this problem, or use {force: true} to disable error checking.Learn more
t/cypress/integration/Agreements_spec.ts:181:38
  179 |         // Click the button in the toolbar
  180 |         cy.visit("/cgi-bin/koha/erm/agreements");
> 181 |         cy.contains("New agreement").click();

This is what I am seeing: https://snipboard.io/Ts2hq8.jpg

Timed out retrying after 4050ms: cy.click() failed because this element:

<a href="/cgi-bin/koha/erm/agreements" class="">...</a>

is being covered by another element:

<div class="modal_centered" data-v-018a4733="">...</div>

Fix this problem, or use {force: true} to disable error checking.Learn more
t/cypress/integration/Agreements_spec.ts:141:55
  139 |         });
  140 |         cy.visit("/cgi-bin/koha/erm/erm.pl");
> 141 |         cy.get("#navmenulist").contains("Agreements").click();
      |                                                       ^
  142 |         cy.get("main div[class='dialog alert']").contains(
  143 |             /Something went wrong/
  144 |         );

It certainly happens because the main "loading" modal is on top of everything, we haven't retrieved the vendors and AVs and so the modal is still on top when we are trying to continue the tests.

Created attachment 149001 [details] [review]
Bug 33370: Intercept and wait for avs and vendors GETs

I think this patch is fixing the problem, but it feels weird to have to add the wait statement to after all our visit calls. Is there another better way?

Maybe we could overload cy.visit to not have to add the wait statement all over the tests (?)

This seems like a decent solution to the problem.

(In reply to Jonathan Druart from comment #8)
> Maybe we could overload cy.visit to not have to add the wait statement all
> over the tests (?)

What do you mean by overload cy.visit?

(In reply to Pedro Amorim from comment #9)
> This seems like a decent solution to the problem.
> 
> (In reply to Jonathan Druart from comment #8)
> > Maybe we could overload cy.visit to not have to add the wait statement all
> > over the tests (?)
> 
> What do you mean by overload cy.visit?

https://docs.cypress.io/api/cypress-api/custom-commands#Overwrite-Existing-Commands

But actually it might be better to create our own visit command instead.

Created attachment 149119 [details] [review]
Bug 33370: Create a new visit_and_wait command

It seems easier and cleaner to have our own visit_and_wait command that
will intercept the 2 queries (for avs and vendors) and then do the wait.

This is needed for the first hit to erm module.

This looks much cleaner. Good choice.

Note that the command is named "visit_and_wait", but it is actually ERM specific, it should be put in a "erm" subdirectory or renamed.

Titles is failing consistently with

    <testcase name="Title CRUD operations Add title" time="0.0000" classname="Add title">      <failure message="Timed out retrying after 4000ms: Expected to find element: `main div[class=&apos;dialog alert&apos;]`, but never found it." type="AssertionError"><![CDATA[AssertionError: Timed out retrying after 4000ms: Expected to find element: `main div[class='dialog alert']`, but never found it.

when run from the command line.
But passes when run from the Cypress UI. I think it has something to do with the size of the window (the alert appears at the top, and is not visible when we submit the form).

(In reply to Jonathan Druart from comment #14)
> Titles is failing consistently with
> 
>     <testcase name="Title CRUD operations Add title" time="0.0000"
> classname="Add title">      <failure message="Timed out retrying after
> 4000ms: Expected to find element: `main div[class=&apos;dialog
> alert&apos;]`, but never found it."
> type="AssertionError"><![CDATA[AssertionError: Timed out retrying after
> 4000ms: Expected to find element: `main div[class='dialog alert']`, but
> never found it.
> 
> when run from the command line.
> But passes when run from the Cypress UI. I think it has something to do with
> the size of the window (the alert appears at the top, and is not visible
> when we submit the form).

Moved to bug 33400.

Wondering if this actually needed if we have bug 33408, and specifically this change in Main.vue:

-    <div v-if="ERMModule">
+    <div v-if="initialized">

(In reply to Jonathan Druart from comment #16)
> Wondering if this actually needed if we have bug 33408, and specifically
> this change in Main.vue:
> 
> -    <div v-if="ERMModule">
> +    <div v-if="initialized">

Giving this one a try and closing this one for now.

*** This bug has been marked as a duplicate of bug 33408 ***