Bug 37620 - Fix randomly failing tests for cypress/integration/InfiniteScrollSelect_spec.ts
Summary: Fix randomly failing tests for cypress/integration/InfiniteScrollSelect_spec.ts
Status: Needs documenting
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Matt Blenkinsop
QA Contact: Testopia
URL:
Keywords: RM_priority
Depends on: 32474
Blocks: 25551
  Show dependency treegraph
 
Reported: 2024-08-12 13:23 UTC by Katrin Fischer
Modified: 2024-10-21 12:23 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
24.11.00,24.05.04
Circulation function:


Attachments
Bug 37620: Improve cypress test to reduce random failures (7.66 KB, patch)
2024-08-19 13:20 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 37620: Improve cypress test to reduce random failures (7.66 KB, patch)
2024-08-27 09:20 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 37620: Fix Total-Count headers (3.58 KB, patch)
2024-08-27 09:21 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 37620: Fix Total-Count headers (3.64 KB, patch)
2024-08-27 09:46 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Katrin Fischer 2024-08-12 13:23:49 UTC
One of the Cypress tests keeps failing randomly on D12. Last today, run #2999

Should correctly display labels.Infinite scroll Should correctly display labels

Timed out retrying after 10000ms: `cy.wait()` timed out waiting `10000ms` for the 1st request to the route: `resetDropdown`. No request ever occurred.  https://on.cypress.io/wait

CypressError: Timed out retrying after 10000ms: `cy.wait()` timed out waiting `10000ms` for the 1st request to the route: `resetDropdown`. No request ever occurred.

https://on.cypress.io/wait
    at cypressErr (http://localhost:8081/__cypress/runner/cypress_runner.js:75187:18)
    at Object.errByPath (http://localhost:8081/__cypress/runner/cypress_runner.js:75242:10)
    at checkForXhr (http://localhost:8081/__cypress/runner/cypress_runner.js:134821:84)
    at <unknown> (http://localhost:8081/__cypress/runner/cypress_runner.js:134844:28)
    at tryCatcher (http://localhost:8081/__cypress/runner/cypress_runner.js:1807:23)
    at Promise.attempt.Promise.try (http://localhost:8081/__cypress/runner/cypress_runner.js:4315:29)
    at whenStable (http://localhost:8081/__cypress/runner/cypress_runner.js:143192:68)
    at <unknown> (http://localhost:8081/__cypress/runner/cypress_runner.js:143133:14)
    at tryCatcher (http://localhost:8081/__cypress/runner/cypress_runner.js:1807:23)
    at Promise._settlePromiseFromHandler (http://localhost:8081/__cypress/runner/cypress_runner.js:1519:31)
    at Promise._settlePromise (http://localhost:8081/__cypress/runner/cypress_runner.js:1576:18)
    at Promise._settlePromise0 (http://localhost:8081/__cypress/runner/cypress_runner.js:1621:10)
    at Promise._settlePromises (http://localhost:8081/__cypress/runner/cypress_runner.js:1701:18)
    at Promise._fulfill (http://localhost:8081/__cypress/runner/cypress_runner.js:1645:18)
    at <unknown> (http://localhost:8081/__cypress/runner/cypress_runner.js:5450:46)
From Your Spec Code:
    at Context.eval (webpack://koha/./t/cypress/integration/InfiniteScrollSelect_spec.ts:248:11)
Comment 1 Jonathan Druart 2024-08-15 13:42:21 UTC
Matt, the tests are failing quite consistently for me now. They almost fail all the time, but the error can be different. Always related to a time out.

I have noticed a couple of things:
* X-Base-Total-Count and X-Total-Count are always 20 when they should be 60 (certainly not the cause of the problem however)

* The "delay" (to prevent too many requests) does not work.

licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25L%25%22%7D%7D
licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25Li%25%22%7D%7D
licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25Lic%25%22%7D%7D
licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25Lice%25%22%7D%7D
licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25Licen%25%22%7D%7D
licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25Licens%25%22%7D%7D
licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25License%25%22%7D%7D
licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25License+%25%22%7D%7D
licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25License+5%25%22%7D%7D
licenses?_per_page=-1&q=%7B%22me.name%22%3A%7B%22like%22%3A%22%25License+50%25%22%7D%7D
Comment 2 Jonathan Druart 2024-08-15 13:49:25 UTC
(In reply to Jonathan Druart from comment #1)
> Matt, the tests are failing quite consistently for me now. They almost fail
> all the time, but the error can be different. Always related to a time out.
> 
> I have noticed a couple of things:
> * X-Base-Total-Count and X-Total-Count are always 20 when they should be 60
> (certainly not the cause of the problem however)
> 
> * The "delay" (to prevent too many requests) does not work.

We are only using setTimeout (so postponing the request), not implementing a throttle function.
Comment 3 Jonathan Druart 2024-08-15 14:08:09 UTC
(In reply to Jonathan Druart from comment #2)
> (In reply to Jonathan Druart from comment #1)
> > Matt, the tests are failing quite consistently for me now. They almost fail
> > all the time, but the error can be different. Always related to a time out.
> > 
> > I have noticed a couple of things:
> > * X-Base-Total-Count and X-Total-Count are always 20 when they should be 60
> > (certainly not the cause of the problem however)
> > 
> > * The "delay" (to prevent too many requests) does not work.
> 
> We are only using setTimeout (so postponing the request), not implementing a
> throttle function.

Not the same place, this is for the scroll, not the filtering.
Comment 4 Matt Blenkinsop 2024-08-19 13:20:59 UTC
Created attachment 170468 [details] [review]
Bug 37620: Improve cypress test to reduce random failures

This patch makes some improvements to the Infinite Scroll cypress test to try and prevent random test failures

- Fixes X-Base-Count and X-Total-Count for intercepted requests
- Makes element selection more precise
- Adds an intercepted request returning the requrired result for the filtered search
- Awaits this request before selecting an element from the dropdown
- Awaits all 10 requests triggered by typing "License 50" rather than just the first one
- Ensures that the select dropdown is closed between operations by clicking into another field to remove focus from the select dropdown

Test plan:
1) yarn cypress run --spec t/cypress/integration/InfiniteScrollSelect_spec.ts
or
1) yarn cypress open and select the InfiniteScrollSelect test from the Cypress GUI
Comment 5 Matt Blenkinsop 2024-08-19 13:23:39 UTC
This should offer a substantial improvement, prior to this the test would fail relatively consistently locally, with this patch it passes every time. It will be interesting to see what happens when this is applied to a full test suite run on a Jenkins node
Comment 6 Katrin Fischer 2024-08-19 13:26:30 UTC
Thanks Matt!
Comment 7 Katrin Fischer 2024-08-19 13:31:49 UTC
Tested on main locally - tests pass. I'll throw it at Jenkins with the next push :)
Comment 8 Katrin Fischer 2024-08-19 15:00:03 UTC
Pushed for 24.11!

Well done everyone, thank you!
Comment 9 Jonathan Druart 2024-08-26 07:06:55 UTC
We should actually throttle here, not request on every key press.

Also I don't think the "40" are correct in this patch, it should be 60 everywhere (even for page 2 we expect 60 rows in total in DB).
Comment 10 Matt Blenkinsop 2024-08-27 09:20:46 UTC
Created attachment 170749 [details] [review]
Bug 37620: Improve cypress test to reduce random failures

This patch makes some improvements to the Infinite Scroll cypress test to try and prevent random test failures

- Fixes X-Base-Count and X-Total-Count for intercepted requests
- Makes element selection more precise
- Adds an intercepted request returning the requrired result for the filtered search
- Awaits this request before selecting an element from the dropdown
- Awaits all 10 requests triggered by typing "License 50" rather than just the first one
- Ensures that the select dropdown is closed between operations by clicking into another field to remove focus from the select dropdown

Test plan:
1) yarn cypress run --spec t/cypress/integration/InfiniteScrollSelect_spec.ts
or
1) yarn cypress open and select the InfiniteScrollSelect test from the Cypress GUI
Comment 11 Matt Blenkinsop 2024-08-27 09:21:43 UTC
Created attachment 170750 [details] [review]
Bug 37620: Fix Total-Count headers
Comment 12 Jonathan Druart 2024-08-27 09:46:19 UTC
Created attachment 170752 [details] [review]
Bug 37620: Fix Total-Count headers

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 13 Katrin Fischer 2024-08-27 09:49:24 UTC
Pushed follow-up patch to main.
Comment 14 Lucas Gass (lukeg) 2024-09-12 21:54:08 UTC
Backported to 24.05.x for upcoming 24.05.04
Comment 15 Fridolin Somers 2024-10-21 12:23:05 UTC
Depends on Bug 32474 not in 23.11.x