Bug 34076 - [ALTERNATE] [DOCS] Automated screenshots using Cypress.io testing framework
Summary: [ALTERNATE] [DOCS] Automated screenshots using Cypress.io testing framework
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: Documentation (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-06-21 12:05 UTC by Jonathan Druart
Modified: 2024-03-27 14:18 UTC (History)
12 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 33956: Use Koha::Biblio->opac_summary_html from opac-user.pl (4.96 KB, patch)
2023-07-14 14:54 UTC, Philip Orr
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2023-06-21 12:05:04 UTC
See also bug 29716 for another implementation but the idea is the same.

We want to use cypress to generate the screenshots of the manual.

See also for a previous discussion - https://lists.katipo.co.nz/pipermail/koha/2018-January/049807.html
Comment 1 Jonathan Druart 2023-06-21 15:02:09 UTC Comment hidden (obsolete)
Comment 2 Jonathan Druart 2023-07-12 15:04:04 UTC
Note that I just retried and was getting an error:

Your configFile is invalid: /e2e/cypress.config.ts

It threw an error when required, check the stack trace below:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /e2e/cypress.config.ts

I had to run `yarn install` (in the Koha src directory).
Comment 3 Jonathan Druart 2023-07-12 15:08:14 UTC
Also note that "3. Run the tests" will fail if no language is installed.
Comment 4 Jonathan Druart 2023-07-12 15:13:42 UTC Comment hidden (obsolete)
Comment 5 Jonathan Druart 2023-07-12 15:22:41 UTC
I have pushed a branch at https://gitlab.com/joubu/Koha/-/commits/cypress-studio

The git log is a bit dirty but at least shows how I reached the current state. Note that this first try (POC) adds code to the Koha repo, but ideally we will certainly want it into its own repo.


The idea is to use cypress-studio |1] to let users without development skills generate steps to reach the DOM element to screenshot (either the whole screen or a specific div). However we will then need developers to polish the generated code.
So far the remote branch only generates 4 screenshots: prefsearch, searchbar-admin, prevcheckout and allowstafftosetvisibilityforguarantor (they are the names of the images in the manual).

On the remote branch you can see "auto" and "manual" commits for those examples. The "auto" are auto-generated by cypress-studio. The "manual" is what needs to be done to make the tests run several times (either restore the state or init it before we run the tests).
Eg. for prevcheckout we need to set the syspref to its original value (hardno) or it will already be "hardyes" and the test will fail to set it to "hardyes" The "manual" step also contain the screenshot part.

Test plan:
0.a Checkout the remote branch
% git remote add joubu https://gitlab.com/joubu/Koha/
% git fetch joubu
% git checkout -b cypress-studio joubu/cypress-studio

0.b start ktd up

1. Generate the necessary built files:
1.a In your host: # will generate node_modules
  % yarn install
1.b Inside ktd:   # should not be needed, but it is needed if ktd was up before you checked out the branch
  % yarn build
1.c Generate the template for other languages, inside ktd:
  % cd misc/translator/
  % perl translate install fr-FR
  % perl translate install fr-CA
  % perl translate install es-ES
Go to Koha, enable the 3 languages (search for the "language" syspref)

2. Run a docker container with the newest version of cypress (Koha is using 9 we want 12)
Go to your Koha src directory and run (on your host then):
% docker run -it \
  -v $PWD:/e2e \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -w /e2e \
  -e DISPLAY \
  --entrypoint cypress \
  --network koha_kohanet \
  cypress/included:12.14.0 open --project .
If you get "No protocol specified" then run (still on the host)
% xhost local:root
And try again the 'docker run' command.
It should open the cypress UI in a new window.
Click "Continue" and you should see: https://snipboard.io/5KawXe.jpg
Click "E2E Testing" (that should be with a green 'Configured').
Select the browser you want to try https://snipboard.io/Aynf64.jpg and click "Start E2E Testing in..."
It will open a new window with the browser. It will list the different test files (found in t/cypress/integration): https://snipboard.io/Lk19cR.jpg

3. Run the tests
Click on Manual/0-language-ui.ts https://snipboard.io/Bd3v4A.jpg
The test should be green
Click on "Specs" top left to open the side menu
Click on 1-sysprefs.ts https://snipboard.io/4iIs2j.jpg
Click on the next 2 tests.
Are they all green? They should.

4. Admire the screenshots
Go to t/cypress/screenshots/ you should see 4 screenshots
en_prefsearch.png
en_searchbar-admin.png
en_prevcheckout.png
en_allowstafftosetvisibilityforguarantor.png

5. Generate translated screenshots
Close the Cypress UI (or ctrl-c in the console where you launched it), and now pass --env KOHA_LANG=fr-FR (to use the French template)
docker run -it \
  -v $PWD:/e2e \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -w /e2e \
  -e DISPLAY \
  --entrypoint cypress \
  --network koha_kohanet \
  cypress/included:12.14.0 open --env KOHA_LANG=fr-FR --project .
And retry 3.
Now you should notice 4 new screenshots, prefixed with "fr-FR_" instead of "en_". They should be showing the French interface.

6. Create a new (easy) test!
Here we are!
Let's start with an easy one, librarylist: https://koha-community.org/manual/23.05/en/html/_images/librarylist.png
So it would be another file, as it's a "library" thing create the following file: `t/cypress/integration/Manual/4-library.ts`
With the following content:
```
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    it("librarylist", function () {
        assert(true);
    });
});
```
Go to the cypress UI (you can use a translated interface or the default/English one) and notice that it detected the new file, click on it. The beforeEach part is only there to login and set the language cookie before each test is run.
The "it" block is doing nothing (apart from running a test that will always pass), but it's where we want our steps to be.
https://snipboard.io/B4YCrK.jpg
Hover your mouse over the "Libraries" line and notice the magic wand icon "Add new test". Tests rerun and you can interact with the UI!
Go to Administation > Libraries. Now the goal is to find the correct element we want to screenshot. We want the "New library" button and the library list.
Right click and locate the 'main' element (between the button and the "Libraries" title for instance).
Click "Add assertion > be visible". Try to make as little "noise" as possible! Every interaction (mouse or keyboard) with the UI will be recorded! However, use the computer like it is your first time: do not use keyboard shortcuts, click on the input before you type, type slowly to make sure everything is recorded, click on the submit button instead of pressing enter, etc.
You can remove the unnecessary steps by clicking the "times-circle" icon next to the step.

When you are ready click "Save commands", enter a title: "librarylist".
Now edit t/cypress/integration/Manual/4-library.ts

You should see a new test added, something like:
    /* ==== Test Created with Cypress Studio ==== */
    it('librarylist', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get(':nth-child(1) > :nth-child(4) > :nth-child(1) > a').click();
        cy.get('main').should('be.visible');
        /* ==== End Cypress Studio ==== */
    });
Remove the first librarylist 'it' (this is not ideal but I didn't manage to find an easy way to not do it for now).

Commit! % git commit -a -m"auto librarylist"
Edit it again replace the line
        cy.get('main').should('be.visible');
With
        cy.get('main').should('be.visible').screenshot("librarylist");
Save.
Look at the screenshot!
Hey! It was not an easy one (I didn't know I promise!)
The screenshot does not show the libraries, because they are fetched in ajax, after the page is loaded.

So edit again and add a "wait_for_ajax" call (that is doing what its named says it does):
(I removed the comment lines)
```
    it('librarylist', function() {
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get(':nth-child(1) > :nth-child(4) > :nth-child(1) > a').click();
        cy.wait_for_ajax();
        cy.get('main').should('be.visible').screenshot("librarylist");
    });
```
Save, look at the screenshot!
Now it is showing the library list!
Commit! `git commit -a -m"manual librarylist"

Do not be afraid, this will be the work for a developer if you are not confident enough with modifying the test file.

You can try a really easy one by yourself: https://koha-community.org/manual/23.05/en/html/_images/newlibrary.png
There is no trick here!
The generated code should be something like:
    it('newlibrary', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get(':nth-child(1) > :nth-child(4) > :nth-child(1) > a').click();
        cy.get('#newbranch').click();
        cy.get('#Aform > :nth-child(1)').should('be.visible');
        /* ==== End Cypress Studio ==== */
    });

7. Let's create a new complex test!
Yes I know, the previous one was not easy finally, but it was not data dependent.
patrondebt - https://koha-community.org/manual/23.05/en/html/_images/patrondebt.png
We need to turn on AllowFineOverride, create a manual fine for a patron, and try to check them an item out.
So click on "2-checkouts", "Checkouts", magic hand, and use the UI. Try to avoid any unnecessary actions! Try to do it perfectly the first time.
Save the commands, run the test: it will fail. Why? Because the pref is now on, and you are trying to turn it on! However you could try your test after a reset_all (with a fresh DB). They should pass in this case, or it means something went wrong when you recorded the tests.
No worries, once again it can be the work of a developer to adjust the tests.

You should have something like that in 2-checkouts.ts
   /* ==== Test Created with Cypress Studio ==== */
    it('patrondebt', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general > .fa').click();
        cy.get('fieldset > [type="text"]').type("AllowFineOverride");
        cy.get('.btn').click();
        cy.get('#pref_AllowFineOverride').select('1');
        cy.get('.save-all').click();
        cy.get('#toplevelmenu > :nth-child(2) > a').click();
        cy.get('#search_patron_filter').clear();
        cy.get('#search_patron_filter').type('edna');
        cy.get('.action > .btn').click();
        cy.get('#menu > ul > :nth-child(3) > a').click();
        cy.get('.manualinvoice > a').click();
        cy.get('#amount').type('42');
        cy.get('[value="save"]').click();
        cy.get('#menu > ul > :nth-child(1) > a').click();
        cy.get('#barcode').clear();
        cy.get('#barcode').type('3999900000001');
        cy.get('#circ_circulation_issue > .btn').click();
        cy.get('#circ_needsconfirmation').should('be.visible');
        /* ==== End Cypress Studio ==== */
    });

We want to modify this code to define the conditions to make it always pass: set the syspref to 0 and delete the fines we created for the patron.
See the comment in the code below:
    it.only('patrondebt', function() {
        cy.set_syspref("AllowFineOverride", "0"); // We set the syspref AllowFineOverride to 0
        cy.query( // We remove the accountlines for the patron we are going to use (edna)
            "DELETE FROM accountlines WHERE borrowernumber=5"
        );
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general > .fa').click();
        cy.get('fieldset > [type="text"]').type("AllowFineOverride");
        cy.get('.btn').click();
        cy.get('#pref_AllowFineOverride').select('1');
        cy.get('.save-all').click();
        cy.get('#toplevelmenu > :nth-child(2) > a').click();
        cy.get('#search_patron_filter').clear();
        cy.get('#search_patron_filter').type('edna');
        cy.get('.action > .btn').click();
        cy.get('#menu > ul > :nth-child(3) > a').click();
        cy.get('.manualinvoice > a').click();
        cy.get('#amount').clear();
        cy.get('#amount').type('42');
        cy.get('[value="save"]').click();
        cy.get("#menu > ul > :nth-child(1) > a").click();
        cy.get("#barcode").clear();
        cy.get("#barcode").type("3999900000001");
        cy.get("#circ_circulation_issue > .btn").click();
        cy.get('#circ_needsconfirmation').should('be.visible').screenshot("patrondebt"); // We add the screenshot call!
    });

Look at the screenshot!
Run the tests again, they are still passing and the screenshot is still the same! All work as expected.

8. Read the doc! (to be polite)
See [1]

Summary:
We could try cypress-studio and see how non-dev people are happy with it. But the code generated will be, in my opinion, unmaintainable (ie. not robust).
See the "Move to commands" commit, where we actually want to replace the generated code with something more readable and robust.
I personally think that the structure of the test files is quite easy to understand, and with a bit more of command (like set_syspref, checkin, checkout) we could make things easier to write, way more easier than using cypress studio, but... I am a developer, this is certainly biased...

There are several things to discuss here. Who is onboard? How the syntax read to non-tech people?
Where do we move the code? When are the screenshots generated? How do we interact with the koha-manual project? Which data are we using? Should we always use the same patron, the same biblio, the same item?

I will be happy to help and work on this. Send me your feedback! :)

Note: The librarylist, newlibrary and patrondebt tests are available on a separate remote branch: cypress-studio-more (still in my gitlab repo)

[1] https://docs.cypress.io/guides/references/cypress-studio
Comment 6 Philip Orr 2023-07-13 10:57:57 UTC
Some notes from my testing. I'm not a dev and have no experience with git for that matter so please correct me if I say something stupid.

All of this worked fine for me:
---
>>0.a Checkout the remote branch
>>% git remote add joubu https://gitlab.com/joubu/Koha/
>>% git fetch joubu
>>% git checkout -b cypress-studio joubu/cypress-studio
---
Note that it should be done in your git/Koha/ directory, that is, using your terminal. Not in the ktd shell. :-)

I got an error here:
---
>>0.b start ktd up
---

The exact error message was:
koha-koha-1       | [tsl] ERROR in /kohadevbox/koha/cypress.config.ts(10,9)
koha-koha-1       |       TS2322: Type '{ setupNodeEvents(on: any, config: any): any; experimentalStudio: true; baseUrl: string; specPattern: string; supportFile: string; env: { db: { host: string; user: string; password: string; database: string; }; }; }' is not assignable to type 'Partial<Omit<ResolvedConfigOptions, TestingType>>'.
koha-koha-1       |   Object literal may only specify known properties, and 'setupNodeEvents' does not exist in type 'Partial<Omit<ResolvedConfigOptions, TestingType>>'.
koha-koha-1       | ts-loader-default_0c5a263502dc9404
koha-koha-1       |
koha-koha-1       | ERROR in /kohadevbox/koha/cypress.config.ts
koha-koha-1       | 28:8-31:9
koha-koha-1       | [tsl] ERROR in /kohadevbox/koha/cypress.config.ts(28,9)
koha-koha-1       |       TS2322: Type '{ devServer: { framework: string; bundler: string; }; }' is not assignable to type 'Partial<Omit<ResolvedConfigOptions, TestingType>>'.
koha-koha-1       |   Object literal may only specify known properties, and 'devServer' does not exist in type 'Partial<Omit<ResolvedConfigOptions, TestingType>>'.
koha-koha-1       | ts-loader-default_0c5a263502dc9404
koha-koha-1       |
koha-koha-1       | webpack 5.74.0 compiled with 2 errors in 5362 ms
koha-koha-1       | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
koha-koha-1       | error Command failed with exit code 1.
koha-koha-1       | error Command failed with exit code 1.
koha-koha-1       | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
koha-db-1         | 2023-07-13 10:15:21 12 [Warning] Aborted connection 12 to db: 'koha_kohadev' user: 'koha_kohadev' host: '172.23.0.4' (Got an error reading communication packets)
koha-db-1         | 2023-07-13 10:15:21 14 [Warning] Aborted connection 14 to db: 'koha_kohadev' user: 'koha_kohadev' host: '172.23.0.4' (Got an error reading communication packets)
koha-db-1         | 2023-07-13 10:15:21 13 [Warning] Aborted connection 13 to db: 'koha_kohadev' user: 'koha_kohadev' host: '172.23.0.4' (Got an error reading communication packets)
koha-koha-1 exited with code 1

I fixed it and I think there were two problems here.

First, I apparently hadn't installed yarn yet. Fixed with:

npm install --global yarn

Then I tried running "yarn install" in the terminal. I got the message:

The engine "node" is incompatible with this module. Expected version "8 || 9 || 10 || 11 || 12 || 13 || 14 || 15 || 16 || 17 || 18". Got "20.3.1"

So it turns out I had Node.js version 20 already installed. You can install a previous version e.g. 18 if you get the same error. I'm using nvm. So I used the following commands:

nvm current node -v  # Checks your current version
nvm install <version>  # In this case I went for 18: nvm install 18

After that running "yarn install" worked correctly, though it warned about missing dependencies.

[3/4] Linking dependencies...
warning " > bootstrap@4.6.2" has unmet peer dependency "jquery@1.9.1 - 3".
warning " > bootstrap@4.6.2" has unmet peer dependency "popper.js@^1.16.1".

Anyway after all that, running "ktd up" works again without any error messages.
Comment 7 Philip Orr 2023-07-13 11:18:53 UTC
A note for Windows users: In step 4, If you're using ktd on wsl, your screenshots will be in your wsl subsystem. If you want to look at them in windows, you can do that easily in Windows by opening your Explorer (win+e) and you should see "Linux"->"Debian" on the left side of your explorer. In there the screenshots will be in:
\\wsl.localhost\Debian\home\(whatever your user name ist)\git\koha\t\cypress\screenshots
Comment 8 Philip Orr 2023-07-13 11:37:48 UTC Comment hidden (obsolete)
Comment 9 Philip Orr 2023-07-13 11:46:35 UTC Comment hidden (obsolete)
Comment 10 Jonathan Druart 2023-07-13 13:56:56 UTC
(In reply to Philip Orr from comment #8)
> Note for clarity's sake:
> In step 6:
> >>Commit! % git commit -a -m"auto librarylist"
> Should be done in ktd --shell.

You can do this on your host as well.

(In reply to Philip Orr from comment #9)
> In step 6, at this step:
> >>So edit again and add a "wait_for_ajax" call (that is doing what its named says it does):
> I get an error running the test after adding the cy.wait_for_ajax call:
> 
> TypeError
> cy.wait_for_ajax is not a function
> t/cypress/integration/Manual/4-library.ts:12:12
> 
>   10 |         cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) >
> .icon_general').click();
>   11 |         cy.get(':nth-child(1) > :nth-child(4) > :nth-child(1) >
> a').click();
> > 12 |         cy.wait_for_ajax();
>      |            ^
>   13 |         cy.get('main').should('be.visible').screenshot("librarylist");
>   14 |         /* ==== End Cypress Studio ==== */
>   15 |     });

I've updated the remote branch to include 2 missing commits.
Comment 11 Caroline Cyr La Rose 2023-07-13 16:33:05 UTC
The error "No protocol specified" from step 2 is this

```
Cypress failed to start.

This may be due to a missing library or dependency. https://on.cypress.io/required-dependencies

Please refer to the error below for more details.

----------

Authorization required, but no authorization protocol specified
[17:0713/160502.554239:ERROR:ozone_platform_x11.cc(240)] Missing X server or $DISPLAY
[17:0713/160502.554617:ERROR:env.cc(255)] The platform failed to initialize.  Exiting.

----------

Platform: linux-x64 (Debian - 11.6)
Cypress Version: 12.14.0
```
Comment 12 Philip Orr 2023-07-14 10:10:05 UTC
Thanks for the update. After a git pull step 6 works for me now, no problems.

I'm getting a different error now in part 7:
>>We want to modify this code to define the conditions to make it always pass: >>set the syspref to 0 and delete the fines we created for the patron.
>>See the comment in the code below:
>>    it.only('patrondebt', function() {
>>        cy.set_syspref("AllowFineOverride", "0"); // We set the syspref >>AllowFineOverride to 0
>>        cy.query( // We remove the accountlines for the patron we are going to use (edna)
>>...

I tried it first with my own cypress-studio-generated code and then tried it again with this code:

```
    it.only('patrondebt', function() {
        cy.set_syspref("AllowFineOverride", "0"); // set syspref allowfineoverride to 0
        cy.query( //we remove the accountlines for patron we will use (edna
            "DELETE FROM accountlines WHERE borrowernumber=5"
        );
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get('fieldset > [type="text"]').type('AllowFineOverride');
        cy.get('.btn').click();
        cy.get('#pref_AllowFineOverride').select('1');
        cy.get('.save-all').click();
        cy.get('img').click();
        cy.get('#findborrower').clear();
        cy.get('#findborrower').type('lisa charles{enter}');
        cy.get('#patronsearch > button').click();
        cy.get('#menu > ul > :nth-child(3) > a').click();
        cy.get('.manualinvoice > a').click();
        cy.get('#type').select('MANUAL');
        cy.get('#amount').clear('5');
        cy.get('#amount').type('5.00');
        cy.get('[value="save"]').click();
        cy.get('#menu > ul > :nth-child(1) > a').click();
        cy.get('#barcode').clear('39999000005615');
        cy.get('#barcode').type('39999000005615{enter}');
        cy.get('#circ_circulation_issue > .btn').click();
        cy.get('#circ_needsconfirmation').should('be.visible').screenshot("patrondebt");
    });
});
```

In both cases I got an error running the test:
```
cy.select() failed because this element is disabled:

<select name="pref_AllowFineOverride" id="pref_AllowFineOverride" class="preference preference-choice">...</select>
```
Comment 13 Philip Orr 2023-07-14 10:11:37 UTC Comment hidden (obsolete)
Comment 14 Philip Orr 2023-07-14 14:47:36 UTC
OK, got it working. Apparently the cy.select part was trying to select
an option too soon when it hadn't been loaded yet?
Anyway just adding cy.wait(2000); before selecting the option in the syspref 
fixed it for me.
```
    it("patrondebt", function () {
        cy.set_syspref("AllowFineOverride", "0");
        cy.query("DELETE FROM accountlines WHERE borrowernumber=5");
        cy.get(
            ":nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general > .fa"
        ).click();
        cy.get('fieldset > [type="text"]').type("AllowFineOverride");
        cy.get(".btn").click();
        cy.wait(2000);
        cy.get("#pref_AllowFineOverride").select("1");
        cy.get(".save-all").click();
        cy.get("#toplevelmenu > :nth-child(2) > a").click();
        cy.get("#search_patron_filter").clear();
        cy.get("#search_patron_filter").type("edna");
        cy.get(".action > .btn").click();
        cy.get("#menu > ul > :nth-child(3) > a").click();
        cy.get(".manualinvoice > a").click();
        cy.get("#amount").clear();
        cy.get("#amount").type("42");
        cy.get('[value="save"]').click();
        cy.get("#menu > ul > :nth-child(1) > a").click();
        cy.get("#barcode").clear();
        cy.get("#barcode").type("3999900000001");
        cy.get("#circ_circulation_issue > .btn").click();
        cy.get("#circ_needsconfirmation")
            .should("be.visible")
            .screenshot("patrondebt");
    });
```
Comment 15 Philip Orr 2023-07-14 14:54:16 UTC Comment hidden (obsolete)
Comment 16 Philip Orr 2023-07-14 15:01:28 UTC
Didn't mean to attach a patch, just to sign off.. Sorry!
Anyway everything worked for me and imo it wasn't too complicated for non-devs like me to use. See the above comments for more info.
Comment 17 Caroline Cyr La Rose 2023-07-20 16:58:53 UTC
With Jonathan's help, I was finally able to run KTD and Cypress. I don't know/remember all the problems we ran into, but I apparently have to delete /kohadevbox/Cypress from the docker (or something) because it doesn't work on my computer (because I'm not 1000 or something).

    docker exec -it koha-koha-1 bash

    rm -rf /kohadevbox/Cypress

And I also needed to disable Wayland. Cypress opened with the long multi-line command in step 2, but the window was just white with nothing else. Jonathan found this https://github.com/cypress-io/cypress/issues/20891#issuecomment-1375410342 I needed to edit the /etc/gdm3/custom.conf file to uncomment #WaylandEnable=false and then restart my computer. So far I haven't had side-effects from this (crossed fingers).

I also ran in the same problem as Philip on step 0b (see comment #6). I had to update node. I had to install nvm. I tried a lot of different things, cursed a little bit too, so I'm not sure what worked in the end, but here are the commands I did (found here https://blog.hubspot.com/website/update-node-js), insert curses anywhere you see fit

    node --version 

    curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash

    source ~/.bashrc

    nvm -v

    nvm install 18

    yarn install


On step 5, all the tests end up red (they were all green in step 3), and I only have 3 screenshots. I have a lot of languages installed on my Koha, I don't know if that's why. I only enabled the 4 languages in the preferences on step 1c (en, fr-CA, fr-FR, es-ES), but I have like 6 or 7 language options in total.

Error in 0-language-ui.ts https://snipboard.io/NZo0Lm.jpg
Error in 1-sysprefs.ts https://snipboard.io/im6WlB.jpg
Error in 2-checkouts.ts https://snipboard.io/OJ6wVE.jpg
Error in 3-patrons.ts https://snipboard.io/4yj2w8.jpg

On step 6, removing the "first libraries 'it'" is lines 7 to 9 (it wasn't clear to me where the element ended, so I had to go by trial and error).

  1 describe("Libraries", function () {
  2     beforeEach(() => {
  3         cy.login();
  4         cy.set_cookie_lang();
  5     });
  6 
  7     it("librarylist", function () {
  8         assert(true);
  9     });
 10 
 11     /* ==== Test Created with Cypress Studio ==== */
 12     it('librarylist', function() {
 13         /* ==== Generated with Cypress Studio ==== */
 14         cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
 15         cy.get(':nth-child(1) > :nth-child(4) > :nth-child(1) > a').click();
 16         cy.get('main').should('be.visible');
 17         /* ==== End Cypress Studio ==== */
 18     });
 19 });

 In step 6 still, the first commit command didn't work for me, I think because the file did not exist before (nothing added to commit but untracked files present (use "git add" to track)), so I did

    git add 4-library.ts

    git commit -m "auto librarylist"

I did this in ktd --shell, but I think it might have worked in my host too, I'm not sure.

Haven't tried step 7 yet...
Comment 18 Caroline Cyr La Rose 2023-07-20 19:51:38 UTC
I redid step 5 after removing all the additional languages in my Koha and they were all green!

I don't remember exactly which command ended up working, but I think it was

git clean -fxd
Comment 19 Jonathan Druart 2023-07-21 12:38:05 UTC
(In reply to Caroline Cyr La Rose from comment #17)
> On step 5, all the tests end up red (they were all green in step 3), and I
> only have 3 screenshots. I have a lot of languages installed on my Koha, I
> don't know if that's why. I only enabled the 4 languages in the preferences
> on step 1c (en, fr-CA, fr-FR, es-ES), but I have like 6 or 7 language
> options in total.
> 
> Error in 0-language-ui.ts https://snipboard.io/NZo0Lm.jpg
> Error in 1-sysprefs.ts https://snipboard.io/im6WlB.jpg
> Error in 2-checkouts.ts https://snipboard.io/OJ6wVE.jpg
> Error in 3-patrons.ts https://snipboard.io/4yj2w8.jpg

Problems can appear with translated interfaces if the templates have not been "installed" (ie. updated) after the "git checkout"

> On step 6, removing the "first libraries 'it'" is lines 7 to 9 (it wasn't
> clear to me where the element ended, so I had to go by trial and error).
> 
>   1 describe("Libraries", function () {
>   2     beforeEach(() => {
>   3         cy.login();
>   4         cy.set_cookie_lang();
>   5     });
>   6 
>   7     it("librarylist", function () {
>   8         assert(true);
>   9     });
>  10 
>  11     /* ==== Test Created with Cypress Studio ==== */
>  12     it('librarylist', function() {
>  13         /* ==== Generated with Cypress Studio ==== */
>  14         cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) >
> .icon_general').click();
>  15         cy.get(':nth-child(1) > :nth-child(4) > :nth-child(1) >
> a').click();
>  16         cy.get('main').should('be.visible');
>  17         /* ==== End Cypress Studio ==== */
>  18     });
>  19 });
> 
>  In step 6 still, the first commit command didn't work for me, I think
> because the file did not exist before (nothing added to commit but untracked
> files present (use "git add" to track)), so I did
> 
>     git add 4-library.ts
> 
>     git commit -m "auto librarylist"
> 
> I did this in ktd --shell, but I think it might have worked in my host too,
> I'm not sure.
> 
> Haven't tried step 7 yet...

yes, git commands can be done in ktd or on the host.
Comment 20 Jonathan Druart 2023-07-21 12:42:05 UTC
(In reply to Caroline Cyr La Rose from comment #17)
>  In step 6 still, the first commit command didn't work for me, I think
> because the file did not exist before (nothing added to commit but untracked
> files present (use "git add" to track)), so I did
> 
>     git add 4-library.ts
> 
>     git commit -m "auto librarylist"

Correct, good catch! I am not sure what's the best solution to adjust the test plan... :-/
Comment 21 Caroline Cyr La Rose 2023-07-21 13:52:21 UTC
(In reply to Jonathan Druart from comment #20)
> (In reply to Caroline Cyr La Rose from comment #17)
> >  In step 6 still, the first commit command didn't work for me, I think
> > because the file did not exist before (nothing added to commit but untracked
> > files present (use "git add" to track)), so I did
> > 
> >     git add 4-library.ts
> > 
> >     git commit -m "auto librarylist"
> 
> Correct, good catch! I am not sure what's the best solution to adjust the
> test plan... :-/

I think this command in two steps would always work, whether the file already exists or not (it's the way I always do it...). I know developers prefer shorthand versions ;) but this would work universally. Git "experts" would know to use the shorthand and wouldn't have to follow step by step each command. If the test plan is aimed at non developers and the purpose is to give us an easy step by step to follow, I would say to write it as two commands.

I was only mentioning it in case a git novice stumbled upon it and was stuck there. As you know, I'm often stuck when commands don't behave exactly as they're supposed to :)

Maybe you don't have to change the test plan right away and keep it in your back pocket for when writing the actual how-to?
Comment 22 Jonathan Druart 2023-07-21 14:14:38 UTC
Yes, add+commit is better.

I meant: I don't know how keep an up-to-date version of the test plan. Adding a new comment here does not feel very optimal.
Comment 23 Caroline Cyr La Rose 2023-07-21 14:19:01 UTC
(In reply to Jonathan Druart from comment #22)
> Yes, add+commit is better.
> 
> I meant: I don't know how keep an up-to-date version of the test plan.
> Adding a new comment here does not feel very optimal.

Can you create a new comment with the updated test plan and obsolete the old one? Like for test plans in patches? I usually start from the bottom and find the last test plan, as that one is likely the most up to date.
Comment 24 Aude Charillon 2023-07-24 14:00:36 UTC
My turn to have a go... I feel like we're doing this in order of who's most comfortable with these types of tool, and I'll readily admit that I'm definitely at the bottom of the ladder! :-D  And to continue the metaphor, the first rung was high for me, partly because I don't even have the background knowledge to understand the vocabulary in the instructions. Like, I had to ask where I was supposed to be exactly when Joubu says "in the host" compared to "inside ktd" (yes, I'm that low).

I got an error in step 0.b, which sounds similar to what Philip and Caroline had. Mine said:

koha-koha-1       | Koha requires mpm_itk to be enabled within Apache in order to run.
koha-koha-1       | Typically this can be enabled with:
koha-koha-1       | 
koha-koha-1       |     sudo a2enmod mpm_itk

Matt had helped me install ktd so he kindly helped me with that one too. He said apparenly node wasn't installed.

I am now stuck at step 2. I get this error:

DevTools listening on ws://127.0.0.1:46611/devtools/browser/1d660389-290b-467c-93d6-9a8b22bc0611
[411:0724/133802.190694:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported.
Missing baseUrl in compilerOptions. tsconfig-paths will be skipped

I had the same error today as on Friday. The good news is, when I tried again today I managed to get back to this step very quickly and without any issues!
Comment 25 Jonathan Druart 2023-07-24 17:08:37 UTC
Aude, can you try:
% node -v
What does it return?
If diferent than 14, (something like v14.21.1) then please try:
% sudo npm install -g n 
% sudo n 14
Comment 26 Caroline Cyr La Rose 2023-08-07 22:05:07 UTC
Ok, it took me a lot of trial and error (around 1.5 hours worth), but I got step 7 done. Look at this gorgeous screenshot! https://snipboard.io/bhyizO.jpg (lol!!)

I tried making a test from scratch to replicate the screenshots in the Administration chapter https://koha-community.org/manual/23.05/en/html/administration.html#libraries

It mostly worked. I ran into some problems.

- For the screenshot showing the library URL in the OPAC (https://koha-community.org/manual/23.05/en/html/_images/librarylink.png), I had to create another test because I couldn't find a way to go to the OPAC in the same test (unless I didn't find the correct way?)

- For the screenshot showing the library link (https://koha-community.org/manual/23.05/en/html/_images/librarylink.png) I cannot get the tabs as well as the content. Also, when I clicked around in Cypress Studio to get the correct html section, I could not find it. I went into the inspector and clicked around there to find the correct div id and changed it manually.

- For the screenshots showing the HTML customization (https://koha-community.org/manual/23.05/en/html/_images/libraryinfo.png and https://koha-community.org/manual/23.05/en/html/_images/opacinfo.png), Cypress studio would not save the text I entered in the body of the customization... I wonder if it's a problem with tiny MCE?

- It's not the end of the world, but I could not take screenshots of partial forms (i.e. https://koha-community.org/manual/23.05/en/html/_images/newlibrary.png and https://koha-community.org/manual/23.05/en/html/_images/librarycontact.png will be together with automatic screenshots) This will necessitate a bit of reorganization in the manual.

- It's also not the end of the world, but I always make the screenshots a tiny bit larger than the div or other html section. For example, this is the screenshot taken with Cypress https://snipboard.io/jSgRZz.jpg , compared to the one in the manual https://koha-community.org/manual/23.05/en/html/_images/librarydetails.png . Notice the sides are not cut as close to the text as the automatic screenshot... I find it's not as... aggressive? I don't know how to describe it, but I find looking at the picture with the tiny bit of space feels calmer. Might just be me being nitpicky...

This is the total of the tests I did for the 4-5 screenshots. It took me about 2 hours to get these. I'm stuck at the HTML customization one.

describe("Administration", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('librarylist', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get(':nth-child(1) > :nth-child(4) > :nth-child(1) > a').click();
        cy.wait_for_ajax();
        cy.get('main').should('be.visible').screenshot("librarylist");
        /* ==== End Cypress Studio ==== */
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(7) > .sorting_1 > a').click();
        cy.get('main').should('be.visible').screenshot("librarydetails");
        /* ==== End Cypress Studio ==== */
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('newlibrary', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get(':nth-child(1) > :nth-child(4) > :nth-child(1) > a').click();
        cy.get('#newbranch').click();
        cy.get('main').should('be.visible').screenshot("newlibrary");
        /* ==== End Cypress Studio ==== */
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('librarylink', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get(':nth-child(1) > :nth-child(4) > :nth-child(1) > a').click();
        cy.get(':nth-child(1) > .actions > a.btn').click();
        cy.get('#branchurl').type('https://www.google.com');
        cy.get('.btn').click();
        /* ==== End Cypress Studio ==== */
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('librarylink2', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.visit('http://kohadev.mydnsname.org:8080/cgi-bin/koha/opac-detail.pl?biblionumber=131');
        cy.get('#holdings_panel').should('be.visible').screenshot("librarylink");
        /* ==== End Cypress Studio ==== */
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('libraryinfo1', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(5) > .icon_general').click();
        cy.get(':nth-child(2) > dl > :nth-child(9) > a').click();
        cy.get('#newentry').click();
        cy.get('#location').select('OpacLibraryInfo');
        cy.get('#branchcode').select('CPL');
        cy.get(':nth-child(3) > .flatpickr_wrapper > .flatpickr').click();
        cy.get('.open > .flatpickr-innerContainer > .flatpickr-rContainer > .flatpickr-days > .dayContainer > [aria-label="August 1, 2023"]').click();
        cy.get('#title_default').type('Closure information');
        cy.get('#submit_form').click();
        /* ==== End Cypress Studio ==== */
    });
});


Other things to think about: 

- I noticed the screenshots generated have the language code before (e.g. en_librarylist.png). I'm not sure how this will be managed in the manual. Currently, we refer to an image file in the source/images/ folder. Will there be a symbolic link (or something) to say librarylist.png in the english manual is en_librarylist.png, and in the fr-CA manual it is fr-CA_librarylist.png, etc? If not, how do we manage all the different images?

- How to manage the tests? One per chapter? One per section? One per image? Do we have a naming convention? We don't really have a naming convention currently, but it would be good to think about it if we are going to have thousands of new files to manage. 

We don't need answers for this right away, but it's just things I thought about while doing it.
Comment 27 Jonathan Druart 2023-08-09 13:00:29 UTC
(In reply to Caroline Cyr La Rose from comment #26)

Caroline, you are awesome! :)

I am looking forward to discuss all this with you in a few days in Helsinki, see you soon!

During a talk next Wednesday I will suggest something to move forward with this topic, and maybe recruit new people :)
Comment 28 Caroline Cyr La Rose 2023-10-25 11:50:16 UTC
10.6% of images have been analyzed
50% are easy and 16% are medium

We will start with 10 easy images and add the code here as an attachment. Jonathan will check the generated code and see if it's too much work to clean it up or not.
Comment 29 Philip Orr 2023-12-29 09:49:20 UTC
note for future me:
If after fetching from joubu you get an error message in Koha:
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Unknown column 'lang' in 'where clause' at /kohadevbox/koha/Koha/Objects.pm line 421
just restart ktd (ktd down, then ktd up) and that should fix it.
Comment 30 Katrin Fischer 2023-12-29 10:07:25 UTC
(In reply to Philip Orr from comment #29)
> note for future me:
> If after fetching from joubu you get an error message in Koha:
> DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st
> execute failed: Unknown column 'lang' in 'where clause' at
> /kohadevbox/koha/Koha/Objects.pm line 421
> just restart ktd (ktd down, then ktd up) and that should fix it.

A restart_all might also do the trick if you want to try that first.
Comment 31 Philip Orr 2023-12-29 10:11:28 UTC
(In reply to Katrin Fischer from comment #30)
> (In reply to Philip Orr from comment #29)
> > note for future me:
> > If after fetching from joubu you get an error message in Koha:
> > DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st
> > execute failed: Unknown column 'lang' in 'where clause' at
> > /kohadevbox/koha/Koha/Objects.pm line 421
> > just restart ktd (ktd down, then ktd up) and that should fix it.
> 
> A restart_all might also do the trick if you want to try that first.

I did try that, but it didn't work for me. Could just be PEBKAC though, I might have done something in the wrong order. :-)
Comment 32 Philip Orr 2023-12-29 14:19:54 UTC
so, I did 10 random easy screenshots from the spreadsheet. It all worked. Here is the code:
(Please tell me if I should add this as an attachment instead of as a comment!)

Screenshot for: About -> perl modules
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('aboutperlmodules', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(7) > .icon_general').click();
        cy.get('#perl-tab > span').click();
        cy.get('.tab-content').should('be.visible').screenshot("aboutperlmodules");
        /* ==== End Cypress Studio ==== */
    });
});
```

Screenshot for: Acquisitions -> Add Basket
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('addbasket', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(3) > .icon_general').click();
        cy.get('p > .btn').click();
        cy.get('.btn > .caret').click();
        cy.get('.open > .dropdown-menu > :nth-child(1) > a').click();
        cy.get('.col-md-10').should('be.visible').screenshot("addbasket");
        /* ==== End Cypress Studio ==== */
    });
});
```

Screenshot for New Vendor -> Interfaces
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('newvendor-interfaces', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(3) > .icon_general').click();
        cy.get('.btn-group > .btn').click();
        cy.get('#interfaces > .btn').click();
        cy.get('#interfaces').should('be.visible').screenshot("newvendor-interfaces");
        /* ==== End Cypress Studio ==== */
    });
});
```

Screenshot for Acquisitions -> Currencies
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('currencies', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(3) > .icon_general').click();
        cy.get(':nth-child(6) > :nth-child(3) > a').click();
        cy.get('.page-section').should('be.visible').screenshot("currencies");
        /* ==== End Cypress Studio ==== */
    });
});
```

Screenshot for Administration -> Z39.50/SRU Servers -> New Z39.50 Server
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('newz39', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get(':nth-child(2) > :nth-child(6) > :nth-child(3) > a').click();
        cy.get('[href="/cgi-bin/koha/admin/z3950servers.pl?op=add&type=zed"]').click();
        cy.get('.rows').should('be.visible').screenshot("newz39");
        /* ==== End Cypress Studio ==== */
    });
});
```

Screenshot for Administration -> Debit Types -> Show all debit types
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('alldebittypes', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get(':nth-child(8) > :nth-child(1) > a').click();
        cy.get('#filter_system').click();
        cy.get('#table_debit_types_wrapper').should('be.visible').screenshot("alldebittypes");
        /* ==== End Cypress Studio ==== */
    });
});
```

Screenshot for Administration -> Credit Types -> Show all credit types
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('allcredittypes', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(6) > .icon_general').click();
        cy.get(':nth-child(8) > :nth-child(3) > a').click();
        cy.get('#filter_system').click();
        cy.get('.page-section').should('be.visible').screenshot("credittypes");
        /* ==== End Cypress Studio ==== */
    });
});
```

Screenshot for Acquisitions -> Budgets -> Close Budget (without unreceived orders)
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('nounreceivedorders', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(3) > .icon_general').click();
        cy.get(':nth-child(6) > :nth-child(1) > a').click();
        cy.get('#budgetmenu_1').click();
        cy.get('.btn-group > .dropdown-menu > :nth-child(4) > a').click();
        cy.get('.col-sm-10').should('be.visible').screenshot("nounreceivedorders");
        /* ==== End Cypress Studio ==== */
    });
});
```

Screenshot for Acquisitions -> Budgets -> Main Budget -> Planning -> Months -> Export CSV
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('planningcsv', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(3) > .icon_general').click();
        cy.get(':nth-child(6) > :nth-child(1) > a').click();
        cy.get('.sorting_1 > a').click();
        cy.get(':nth-child(3) > .btn').click();
        cy.get('.open > .dropdown-menu > :nth-child(1) > a').click();
        cy.get('.col-sm-2 > aside:nth-child(1) > form:nth-child(2)').should('be.visible').screenshot("planningcsv");
        /* ==== End Cypress Studio ==== */
    });
});
```
Note: the button "Submit" doesn't show up for some reason, even though it should
be part of the form I selected.

Screenshot for Acquisitions -> Vendors -> My Vendor -> New Contract Form
´´´
describe("Libraries", function () {
    beforeEach(() => {
        cy.login();
        cy.set_cookie_lang();
    });

    /* ==== Test Created with Cypress Studio ==== */
    it('newcontractform', function() {
        /* ==== Generated with Cypress Studio ==== */
        cy.get(':nth-child(2) > .biglinks-list > :nth-child(3) > .icon_general').click();
        cy.get('p > .btn').click();
        cy.get('.suppliername > :nth-child(1)').click();
        cy.get(':nth-child(1) > .btn').click();
        cy.get('.open > .dropdown-menu > :nth-child(2) > a').click();
        cy.get('.page-section').should('be.visible').screenshot("newcontractform");
        /* ==== End Cypress Studio ==== */
    });
});
```