Bug 26031

Summary: www/search_utf8.t is failing randomly and must be removed/replaced
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: Test SuiteAssignee: Mason James <mtj>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: major    
Priority: P5 - low CC: andrewfh, lucas, martin.renvoize, mtj, nick, tomascohen
Version: master   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.11.00
Bug Depends on: 25538    
Bug Blocks: 25551    
Attachments: Bug 26031: Add diagnostic to search_utf8.t
Bug 26031: Add diagnostic to search_utf8.t
Bug 26031: Remove www/search_utf8.t
Bug 26031: add sleep(1) to fix www/search_utf8.t failing randomly
Bug 26031: Remove 4 failing tests from t/db_dependent/www/search_utf8.t
Bug 26031: Remove the failing tests and the diag

Description Jonathan Druart 2020-07-21 08:40:25 UTC
Reported initially on bug 25538, but the proposed patch did not fix the problem.

18:27:17 koha_1       | Warning: <retrieval> element has non-unique value in 'name' key attribute: F at t/db_dependent/www/search_utf8.t line 43.
18:27:17 koha_1       | Warning: <retrieval> element has non-unique value in 'name' key attribute: B at t/db_dependent/www/search_utf8.t line 43.
18:27:50 koha_1       | 
18:27:50 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 259.
18:27:50 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:27:50 koha_1       | #                   'Koha › Catalog › Results of search for 'kw,wrdl: deuteros'  Menu CirculationPatronsSearchAdvanced searchItem searchCartMore ListsCatalogingAcquisitionsAuthoritiesSerialsCourse reservesReportsToolsAdministrationAbout Koha     koha |  Centerville CPL     Logged in as:koha Location: Centerville CPLSet libraryMy accountMy checkoutsLog out HelpYour cart is empty.Enter patron card number or partial name: Scan a barcode to check in: Scan a barcode to renew: Enter search keywords: Check outCheck inRenewSearch the catalogHome › Catalog › Search for 'kw,wrdl: deuteros'  2 result(s) found for 'kw,wrdl: deuteros'.  Select all |  Clear all | Unhighlight Highlight |  Add to cart Add to list New list Place hold Z39.50/SRU search Edit Batch editBatch deleteMerge records Sort RelevancePopularityPopularity (most to least)Popularity (least to most)AuthorAuthor (A-Z)Author (Z-A)Call numberCall number (0-9 to A-Z)Call number (Z-A to 9-0)DatesPublication/Copyright date: newest to oldestPublication/Copyright date: oldest to newestAcquisition date: newest to oldestAcquisition date: oldest to newestTitleTitle (A-Z)Title (Z-A) ResultsLocation 1. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items 2. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items   Refine your searchAvailabilityLimit to currently available itemsTopicsGardeningΚηπουρική学協会. μμ'
18:27:50 koha_1       | #     doesn't match '(?^u:Publisher: Αθήνα)'
18:27:51 koha_1       | 
18:27:51 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 268.
18:27:51 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:27:51 koha_1       | #                   'Koha › Catalog › Results of search for 'kw,wrdl: Αθήνα'  Menu CirculationPatronsSearchAdvanced searchItem searchCartMore ListsCatalogingAcquisitionsAuthoritiesSerialsCourse reservesReportsToolsAdministrationAbout Koha     koha |  Centerville CPL     Logged in as:koha Location: Centerville CPLSet libraryMy accountMy checkoutsLog out HelpYour cart is empty.Enter patron card number or partial name: Scan a barcode to check in: Scan a barcode to renew: Enter search keywords: Check outCheck inRenewSearch the catalogHome › Catalog › Search for 'kw,wrdl: Αθήνα'  2 result(s) found for 'kw,wrdl: Αθήνα'.  Select all |  Clear all | Unhighlight Highlight |  Add to cart Add to list New list Place hold Z39.50/SRU search Edit Batch editBatch deleteMerge records Sort RelevancePopularityPopularity (most to least)Popularity (least to most)AuthorAuthor (A-Z)Author (Z-A)Call numberCall number (0-9 to A-Z)Call number (Z-A to 9-0)DatesPublication/Copyright date: newest to oldestPublication/Copyright date: oldest to newestAcquisition date: newest to oldestAcquisition date: oldest to newestTitleTitle (A-Z)Title (Z-A) ResultsLocation 1. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items 2. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items   Refine your searchAvailabilityLimit to currently available itemsTopicsGardeningΚηπουρική学協会. μμ'
18:27:51 koha_1       | #     doesn't match '(?^u:Publisher: Αθήνα)'
18:27:52 koha_1       | 
18:27:52 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 282.
18:27:52 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:27:52 koha_1       | #                   'Koha online catalog › Results of search for 'deuteros' Skip to main content Koha online Your cart is empty.  Cart   Lists Your listsNo private listsNew list  Welcome, koha  Your accountSearch history   Clear Log out Search Library catalogTitleAuthorSubjectISBNISSNSeriesCall numberGoAdvanced searchCourse reservesAuthority searchTag cloud Libraries ×Log in to your accountLogin:Password:Forgot your password?Don't have an account? Register here.Home ›Results of search for 'deuteros'Refine your searchAvailabilityLimit to currently available items.TopicsGardeningΚηπουρική学協会. μμ Your search returned 2 results.    RelevancePopularity (most to least)Popularity (least to most)Author (A-Z)Author (Z-A)Call number (0-9 to A-Z)Call number (Z-A to 9-0)Publication/Copyright date: Newest to oldestPublication/Copyright date: Oldest to newestAcquisition date: Newest to oldestAcquisition date: Oldest to newestTitle (A-Z)Title (Z-A)  Unhighlight  Highlight|    |     1.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) 2.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) Not finding what you're looking for? Make a purchase suggestionPowered by Koha'
18:27:52 koha_1       | #     doesn't match '(?^u:Publisher: Αθήνα)'
18:27:54 koha_1       | 
18:27:54 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 291.
18:27:54 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:27:54 koha_1       | #                   'Koha online catalog › Results of search for 'Αθήνα' Skip to main content Koha online Your cart is empty.  Cart   Lists Your listsNo private listsNew list  Welcome, koha  Your accountSearch history   Clear Log out Search Library catalogTitleAuthorSubjectISBNISSNSeriesCall numberGoAdvanced searchCourse reservesAuthority searchTag cloud Libraries ×Log in to your accountLogin:Password:Forgot your password?Don't have an account? Register here.Home ›Results of search for 'Αθήνα'Refine your searchAvailabilityLimit to currently available items.TopicsGardeningΚηπουρική学協会. μμ Your search returned 2 results.    RelevancePopularity (most to least)Popularity (least to most)Author (A-Z)Author (Z-A)Call number (0-9 to A-Z)Call number (Z-A to 9-0)Publication/Copyright date: Newest to oldestPublication/Copyright date: Oldest to newestAcquisition date: Newest to oldestAcquisition date: Oldest to newestTitle (A-Z)Title (Z-A)  Unhighlight  Highlight|    |     1.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) 2.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) Not finding what you're looking for? Make a purchase suggestionPowered by Koha'
18:27:54 koha_1       | #     doesn't match '(?^u:Publisher: Αθήνα)'
18:28:00 db_1         | 2020-07-20 16:28:00 325 [Warning] Aborted connection 325 to db: 'koha_kohadev' user: 'koha_kohadev' host: '192.168.48.6' (Got an error reading communication packets)
18:28:33 koha_1       | 
18:28:33 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 259.
18:28:33 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:28:33 koha_1       | #                   'Koha › Catalog › Results of search for 'kw,wrdl: Tòmas'  Menu CirculationPatronsSearchAdvanced searchItem searchCartMore ListsCatalogingAcquisitionsAuthoritiesSerialsCourse reservesReportsToolsAdministrationAbout Koha     koha |  Centerville CPL     Logged in as:koha Location: Centerville CPLSet libraryMy accountMy checkoutsLog out HelpYour cart is empty.Enter patron card number or partial name: Scan a barcode to check in: Scan a barcode to renew: Enter search keywords: Check outCheck inRenewSearch the catalogHome › Catalog › Search for 'kw,wrdl: Tòmas'  2 result(s) found for 'kw,wrdl: Tòmas'.  Select all |  Clear all | Unhighlight Highlight |  Add to cart Add to list New list Place hold Z39.50/SRU search Edit Batch editBatch deleteMerge records Sort RelevancePopularityPopularity (most to least)Popularity (least to most)AuthorAuthor (A-Z)Author (Z-A)Call numberCall number (0-9 to A-Z)Call number (Z-A to 9-0)DatesPublication/Copyright date: newest to oldestPublication/Copyright date: oldest to newestAcquisition date: newest to oldestAcquisition date: oldest to newestTitleTitle (A-Z)Title (Z-A) ResultsLocation 1. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items 2. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items   Refine your searchAvailabilityLimit to currently available itemsAuthorsTòmas, Giovanni Feli...TopicsThomas Aquinas, s.'
18:28:33 koha_1       | #     doesn't match '(?^u:Publisher: Ramòn)'
18:28:33 koha_1       | 
18:28:33 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 268.
18:28:33 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:28:33 koha_1       | #                   'Koha › Catalog › Results of search for 'kw,wrdl: Ramòn'  Menu CirculationPatronsSearchAdvanced searchItem searchCartMore ListsCatalogingAcquisitionsAuthoritiesSerialsCourse reservesReportsToolsAdministrationAbout Koha     koha |  Centerville CPL     Logged in as:koha Location: Centerville CPLSet libraryMy accountMy checkoutsLog out HelpYour cart is empty.Enter patron card number or partial name: Scan a barcode to check in: Scan a barcode to renew: Enter search keywords: Check outCheck inRenewSearch the catalogHome › Catalog › Search for 'kw,wrdl: Ramòn'  2 result(s) found for 'kw,wrdl: Ramòn'.  Select all |  Clear all | Unhighlight Highlight |  Add to cart Add to list New list Place hold Z39.50/SRU search Edit Batch editBatch deleteMerge records Sort RelevancePopularityPopularity (most to least)Popularity (least to most)AuthorAuthor (A-Z)Author (Z-A)Call numberCall number (0-9 to A-Z)Call number (Z-A to 9-0)DatesPublication/Copyright date: newest to oldestPublication/Copyright date: oldest to newestAcquisition date: newest to oldestAcquisition date: oldest to newestTitleTitle (A-Z)Title (Z-A) ResultsLocation 1. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items 2. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items   Refine your searchAvailabilityLimit to currently available itemsAuthorsTòmas, Giovanni Feli...TopicsThomas Aquinas, s.'
18:28:33 koha_1       | #     doesn't match '(?^u:Publisher: Ramòn)'
18:28:35 koha_1       | 
18:28:35 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 282.
18:28:35 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:28:35 koha_1       | #                   'Koha online catalog › Results of search for 'Tòmas' Skip to main content Koha online Your cart is empty.  Cart   Lists Your listsNo private listsNew list  Welcome, koha  Your accountSearch history   Clear Log out Search Library catalogTitleAuthorSubjectISBNISSNSeriesCall numberGoAdvanced searchCourse reservesAuthority searchTag cloud Libraries ×Log in to your accountLogin:Password:Forgot your password?Don't have an account? Register here.Home ›Results of search for 'Tòmas'Refine your searchAvailabilityLimit to currently available items.AuthorsTòmas, Giovanni Feli...TopicsThomas Aquinas, s. Your search returned 2 results.    RelevancePopularity (most to least)Popularity (least to most)Author (A-Z)Author (Z-A)Call number (0-9 to A-Z)Call number (Z-A to 9-0)Publication/Copyright date: Newest to oldestPublication/Copyright date: Oldest to newestAcquisition date: Newest to oldestAcquisition date: Oldest to newestTitle (A-Z)Title (Z-A)  Unhighlight  Highlight|    |     1.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) 2.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) Not finding what you're looking for? Make a purchase suggestionPowered by Koha'
18:28:35 koha_1       | #     doesn't match '(?^u:Publisher: Ramòn)'
18:28:36 koha_1       | 
18:28:36 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 291.
18:28:36 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:28:36 koha_1       | #                   'Koha online catalog › Results of search for 'Ramòn' Skip to main content Koha online Your cart is empty.  Cart   Lists Your listsNo private listsNew list  Welcome, koha  Your accountSearch history   Clear Log out Search Library catalogTitleAuthorSubjectISBNISSNSeriesCall numberGoAdvanced searchCourse reservesAuthority searchTag cloud Libraries ×Log in to your accountLogin:Password:Forgot your password?Don't have an account? Register here.Home ›Results of search for 'Ramòn'Refine your searchAvailabilityLimit to currently available items.AuthorsTòmas, Giovanni Feli...TopicsThomas Aquinas, s. Your search returned 2 results.    RelevancePopularity (most to least)Popularity (least to most)Author (A-Z)Author (Z-A)Call number (0-9 to A-Z)Call number (Z-A to 9-0)Publication/Copyright date: Newest to oldestPublication/Copyright date: Oldest to newestAcquisition date: Newest to oldestAcquisition date: Oldest to newestTitle (A-Z)Title (Z-A)  Unhighlight  Highlight|    |     1.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) 2.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) Not finding what you're looking for? Make a purchase suggestionPowered by Koha'
18:28:36 koha_1       | #     doesn't match '(?^u:Publisher: Ramòn)'
18:28:41 db_1         | 2020-07-20 16:28:41 338 [Warning] Aborted connection 338 to db: 'koha_kohadev' user: 'koha_kohadev' host: '192.168.48.6' (Got an error reading communication packets)
18:29:14 koha_1       | 
18:29:14 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 259.
18:29:14 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:29:14 koha_1       | #                   'Koha › Catalog › Results of search for 'kw,wrdl: A tiny record'  Menu CirculationPatronsSearchAdvanced searchItem searchCartMore ListsCatalogingAcquisitionsAuthoritiesSerialsCourse reservesReportsToolsAdministrationAbout Koha     koha |  Centerville CPL     Logged in as:koha Location: Centerville CPLSet libraryMy accountMy checkoutsLog out HelpYour cart is empty.Enter patron card number or partial name: Scan a barcode to check in: Scan a barcode to renew: Enter search keywords: Check outCheck inRenewSearch the catalogHome › Catalog › Search for 'kw,wrdl: A tiny record'  2 result(s) found for 'kw,wrdl: A tiny record'.  Select all |  Clear all | Unhighlight Highlight |  Add to cart Add to list New list Place hold Z39.50/SRU search Edit Batch editBatch deleteMerge records Sort RelevancePopularityPopularity (most to least)Popularity (least to most)AuthorAuthor (A-Z)Author (Z-A)Call numberCall number (0-9 to A-Z)Call number (Z-A to 9-0)DatesPublication/Copyright date: newest to oldestPublication/Copyright date: oldest to newestAcquisition date: newest to oldestAcquisition date: oldest to newestTitleTitle (A-Z)Title (Z-A) ResultsLocation 1. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items 2. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items   Refine your searchAvailabilityLimit to currently available itemsTopics😀'
18:29:14 koha_1       | #     doesn't match '(?^u:Publisher: 𠻺tomasito𠻺)'
18:29:15 koha_1       | 
18:29:15 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 268.
18:29:15 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:29:15 koha_1       | #                   'Koha › Catalog › Results of search for 'kw,wrdl: 𠻺tomasito𠻺'  Menu CirculationPatronsSearchAdvanced searchItem searchCartMore ListsCatalogingAcquisitionsAuthoritiesSerialsCourse reservesReportsToolsAdministrationAbout Koha     koha |  Centerville CPL     Logged in as:koha Location: Centerville CPLSet libraryMy accountMy checkoutsLog out HelpYour cart is empty.Enter patron card number or partial name: Scan a barcode to check in: Scan a barcode to renew: Enter search keywords: Check outCheck inRenewSearch the catalogHome › Catalog › Search for 'kw,wrdl: 𠻺tomasito𠻺'  2 result(s) found for 'kw,wrdl: 𠻺tomasito𠻺'.  Select all |  Clear all | Unhighlight Highlight |  Add to cart Add to list New list Place hold Z39.50/SRU search Edit Batch editBatch deleteMerge records Sort RelevancePopularityPopularity (most to least)Popularity (least to most)AuthorAuthor (A-Z)Author (Z-A)Call numberCall number (0-9 to A-Z)Call number (Z-A to 9-0)DatesPublication/Copyright date: newest to oldestPublication/Copyright date: oldest to newestAcquisition date: newest to oldestAcquisition date: oldest to newestTitleTitle (A-Z)Title (Z-A) ResultsLocation 1. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items 2. Material type:  Text No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items   Refine your searchAvailabilityLimit to currently available itemsTopics😀'
18:29:15 koha_1       | #     doesn't match '(?^u:Publisher: 𠻺tomasito𠻺)'
18:29:16 koha_1       | 
18:29:16 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 282.
18:29:16 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:29:16 koha_1       | #                   'Koha online catalog › Results of search for 'A tiny record' Skip to main content Koha online Your cart is empty.  Cart   Lists Your listsNo private listsNew list  Welcome, koha  Your accountSearch history   Clear Log out Search Library catalogTitleAuthorSubjectISBNISSNSeriesCall numberGoAdvanced searchCourse reservesAuthority searchTag cloud Libraries ×Log in to your accountLogin:Password:Forgot your password?Don't have an account? Register here.Home ›Results of search for 'A tiny record'Refine your searchAvailabilityLimit to currently available items.Topics😀 Your search returned 2 results.    RelevancePopularity (most to least)Popularity (least to most)Author (A-Z)Author (Z-A)Call number (0-9 to A-Z)Call number (Z-A to 9-0)Publication/Copyright date: Newest to oldestPublication/Copyright date: Oldest to newestAcquisition date: Newest to oldestAcquisition date: Oldest to newestTitle (A-Z)Title (Z-A)  Unhighlight  Highlight|    |     1.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) 2.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) Not finding what you're looking for? Make a purchase suggestionPowered by Koha'
18:29:16 koha_1       | #     doesn't match '(?^u:Publisher: 𠻺tomasito𠻺)'
18:29:17 koha_1       | 
18:29:17 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 291.
18:29:17 koha_1       | Wide character in print at /usr/share/perl5/Test2/Formatter/TAP.pm line 105.
18:29:17 koha_1       | #                   'Koha online catalog › Results of search for '𠻺tomasito𠻺' Skip to main content Koha online Your cart is empty.  Cart   Lists Your listsNo private listsNew list  Welcome, koha  Your accountSearch history   Clear Log out Search Library catalogTitleAuthorSubjectISBNISSNSeriesCall numberGoAdvanced searchCourse reservesAuthority searchTag cloud Libraries ×Log in to your accountLogin:Password:Forgot your password?Don't have an account? Register here.Home ›Results of search for '𠻺tomasito𠻺'Refine your searchAvailabilityLimit to currently available items.Topics😀 Your search returned 2 results.    RelevancePopularity (most to least)Popularity (least to most)Author (A-Z)Author (Z-A)Call number (0-9 to A-Z)Call number (Z-A to 9-0)Publication/Copyright date: Newest to oldestPublication/Copyright date: Oldest to newestAcquisition date: Newest to oldestAcquisition date: Oldest to newestTitle (A-Z)Title (Z-A)  Unhighlight  Highlight|    |     1.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) 2.  Material type:  Text Availability: No items available :      Save to lists  Add to cart (remove) Not finding what you're looking for? Make a purchase suggestionPowered by Koha'
18:29:17 koha_1       | #     doesn't match '(?^u:Publisher: 𠻺tomasito𠻺)'
18:29:23 koha_1       | # Looks like you failed 12 tests of 99.
18:29:23 db_1         | 2020-07-20 16:29:23 350 [Warning] Aborted connection 350 to db: 'koha_kohadev' user: 'koha_kohadev' host: '192.168.48.6' (Got an error reading communication packets)
18:29:23 koha_1       | [16:29:23] t/db_dependent/www/search_utf8.t ........................................ 
18:29:23 koha_1       | Dubious, test returned 12 (wstat 3072, 0xc00)
18:29:23 koha_1       | Failed 12/99 subtests
Comment 1 Jonathan Druart 2020-07-21 08:45:22 UTC
Info from bug 25538:

Still the same failure on 20.05_U18/11 (note that selenium/authentication.t is failing as well)

19.11_U18 #190 (Docker 3)

19.05_D9 #246 (Docker 1)

19.05_D9 #256 (Docker 4)

Master_D9_MDB_Latest #344 (Docker 4)
Comment 2 Jonathan Druart 2020-07-27 09:51:13 UTC
Master_U18 #848
Comment 3 Jonathan Druart 2020-08-03 15:46:10 UTC
Master_D9 #1399
Comment 4 Jonathan Druart 2020-08-03 15:49:12 UTC Comment hidden (obsolete)
Comment 5 Jonathan Druart 2020-08-05 14:45:22 UTC
Master_U16 #39
Comment 6 Jonathan Druart 2020-08-19 15:54:45 UTC
Koha_Master_U20 #73
Comment 7 Jonathan Druart 2020-08-20 09:34:55 UTC
From yesterday, 20.05_D9 #73 (Docker_2)
Comment 8 Nick Clemens 2020-08-20 12:02:08 UTC
It looks like the problem here is that we expect to end up on the details page for the record, but instead we are getting 2 results and ending up on the results page - I cannot see why though
Comment 9 Jonathan Druart 2020-08-20 12:49:52 UTC
(In reply to Nick Clemens from comment #8)
> It looks like the problem here is that we expect to end up on the details
> page for the record, but instead we are getting 2 results and ending up on
> the results page - I cannot see why though

I don't think it's correct:
18:29:17 koha_1       | #   Failed test at t/db_dependent/www/search_utf8.t line 291.
The error is: 'Koha online catalog › Results of blabla' doesn't match '(?^u:Publisher: 𠻺tomasito𠻺)'

Bu:
291     like( $opac_text, qr|Publisher: $publisher|, );
292     $expected_base = q|opac-search.pl\?(idx=&)?q=| . uri_escape_utf8( $publisher );
293     $agent->base_like(qr|$expected_base|, );

=> We expect to be on the search result page
Comment 10 Nick Clemens 2020-08-20 18:21:48 UTC
Looking again - the page returned has search result numbers, but no content:

1. Material type:  Text 
No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items 
2. Material type:  Text 
No holds allowed | Add to cart (remove) | Edit record | Edit items  | OPAC view No items

You can recreate 12 failures by setting all 6 XSLT sysprefs to a non existent file (or just any text, I used "5")

I am guessing another test is altering the XSLT file prefs, or the files cannot be reached for some reason
Comment 11 Jonathan Druart 2020-08-21 08:07:10 UTC
Great findings Nick!
Comment 12 Jonathan Druart 2020-08-21 08:07:28 UTC
Created attachment 108780 [details] [review]
Bug 26031: Add diagnostic to search_utf8.t
Comment 13 Jonathan Druart 2020-08-21 10:29:55 UTC
Created attachment 108798 [details] [review]
Bug 26031: Add diagnostic to search_utf8.t
Comment 14 Jonathan Druart 2020-08-21 10:34:48 UTC
Pushed to master, waiting for the next failure!
Comment 15 Jonathan Druart 2020-08-24 13:48:54 UTC
We got a winner Koha_Master_D10 #332, but the result is disappointing :-/

13:32:50 koha_1       | # ===DIAG===
13:32:50 koha_1       | # XSLTResultsDisplay is default
13:32:50 koha_1       | # /kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/en/xslt/MARC21slim2intranetResults.xsl exists
13:32:50 koha_1       | # ==========

So we are on the wrong track...
Comment 16 Jonathan Druart 2020-09-01 08:01:49 UTC
Koha_20.05_U18 #61 (Docker_3)
Comment 17 Jonathan Druart 2020-09-03 07:55:13 UTC
Master_D10_Deps #64 Docker_3

16:45:29 koha_1       | # ===DIAG===
16:45:29 koha_1       | # XSLTResultsDisplay is default
16:45:29 koha_1       | # /kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/en/xslt/MARC21slim2intranetResults.xsl exists
16:45:29 koha_1       | # ==========
Comment 18 Lucas Gass 2020-09-04 17:31:55 UTC
For me, in 20.05.x, this fails tests with or without the patch. Not backporting until someone advises me on the correct path forward for 20.05.x:

ok 22 - UTF-8 chars are correctly present. Good
Error GETing http://koha:8080: Internal Server Error at t/db_dependent/www/search_utf8.t line 276.
# Looks like your test exited with 255 just after 22.
Comment 19 Jonathan Druart 2020-11-18 16:21:47 UTC
Created attachment 113793 [details] [review]
Bug 26031: Remove www/search_utf8.t

Those tests are not reliable and fail randomly and we did not find the
root of the problem.
I am in favour of removing them, at least for now.
Comment 20 Tomás Cohen Arazi 2020-11-18 16:31:04 UTC
Selenium needs to be replaced altogether as the Perl libraries really don't support newer browsers.
Comment 21 Jonathan Druart 2020-11-18 16:44:27 UTC
search_utf8.t is not using Selenium
Comment 22 Mason James 2020-11-19 03:19:29 UTC
Created attachment 113796 [details] [review]
Bug 26031: add sleep(1) to fix  www/search_utf8.t failing randomly

to test...
 1/ apply patch
 2/ run test
 3/ observe PASS

root@kohadevbox:/kohadevbox/koha# export KOHA_INTRANET_URL="http://kohadev-intra.myDNSname.org:8081"
root@kohadevbox:/kohadevbox/koha# export KOHA_OPAC_URL="http://kohadev.myDNSname.org:8080"
root@kohadevbox:/kohadevbox/koha# prove -v  ./t/db_dependent/www/search_utf8.t
1..99
ok 1 - connect to intranet
ok 2 - login to staff interface
ok 3 - load main page
ok 4 - open tools module
ok 5 - go to stage MARC
ok 6 - uploaded file
ok 7 - upload succeeded
ok 8 - reopen stage MARC page
ok 9 - stage MARC
ok 10 - have job ID
ok 11 - job was completed
ok 12 - reopen stage MARC page at end of upload
ok 13 - stage MARC
ok 14 - view batch
ok 15 - imported records into catalog
ok 16 - got search on intranet
ok 17 - test staff  'kw' search
ok 18 - got search on intranet2
ok 19 - test staff 'kw' search 2
ok 20 - Base is like "(?^u:search.pl\?advsearch=1&idx=kw&q=%CE%91%CE%B8%CE%AE%CE%BD%CE%B1)"
ok 21 - UTF-8 are multi-byte. Good
ok 22 - UTF-8 chars are correctly present. Good
ok 23 - got opac
ok 24 - test opac default search
ok 25 - got opac2
ok 26 - test opac default search 2
ok 27 - Base is like "(?^u:opac-search.pl\?(idx=&)?q=%CE%91%CE%B8%CE%AE%CE%BD%CE%B1)"
ok 28 - Base does not contain an idx
ok 29 - UTF-8 are multi-byte. Good
ok 30 - UTF-8 chars are correctly present. Good
ok 31 - view and clean batch
ok 32 - biblio on intranet
ok 33 - biblio deleted
ok 34 - connect to intranet
ok 35 - login to staff interface
ok 36 - load main page
ok 37 - open tools module
ok 38 - go to stage MARC
ok 39 - uploaded file
ok 40 - upload succeeded
ok 41 - reopen stage MARC page
ok 42 - stage MARC
ok 43 - have job ID
ok 44 - job was completed
ok 45 - reopen stage MARC page at end of upload
ok 46 - stage MARC
ok 47 - view batch
ok 48 - imported records into catalog
ok 49 - got search on intranet
ok 50 - test staff  'kw' search
ok 51 - got search on intranet2
ok 52 - test staff 'kw' search 2
ok 53 - Base is like "(?^u:search.pl\?advsearch=1&idx=kw&q=Ram%C3%B2n)"
ok 54 - UTF-8 are multi-byte. Good
ok 55 - UTF-8 chars are correctly present. Good
ok 56 - got opac
ok 57 - test opac default search
ok 58 - got opac2
ok 59 - test opac default search 2
ok 60 - Base is like "(?^u:opac-search.pl\?(idx=&)?q=Ram%C3%B2n)"
ok 61 - Base does not contain an idx
ok 62 - UTF-8 are multi-byte. Good
ok 63 - UTF-8 chars are correctly present. Good
ok 64 - view and clean batch
ok 65 - biblio on intranet
ok 66 - biblio deleted
ok 67 - connect to intranet
ok 68 - login to staff interface
ok 69 - load main page
ok 70 - open tools module
ok 71 - go to stage MARC
ok 72 - uploaded file
ok 73 - upload succeeded
ok 74 - reopen stage MARC page
ok 75 - stage MARC
ok 76 - have job ID
ok 77 - job was completed
ok 78 - reopen stage MARC page at end of upload
ok 79 - stage MARC
ok 80 - view batch
ok 81 - imported records into catalog
ok 82 - got search on intranet
ok 83 - test staff  'kw' search
ok 84 - got search on intranet2
ok 85 - test staff 'kw' search 2
ok 86 - Base is like "(?^u:search.pl\?advsearch=1&idx=kw&q=%F0%A0%BB%BAtomasito%F0%A0%BB%BA)"
ok 87 - UTF-8 are multi-byte. Good
ok 88 - UTF-8 chars are correctly present. Good
ok 89 - got opac
ok 90 - test opac default search
ok 91 - got opac2
ok 92 - test opac default search 2
ok 93 - Base is like "(?^u:opac-search.pl\?(idx=&)?q=%F0%A0%BB%BAtomasito%F0%A0%BB%BA)"
ok 94 - Base does not contain an idx
ok 95 - UTF-8 are multi-byte. Good
ok 96 - UTF-8 chars are correctly present. Good
ok 97 - view and clean batch
ok 98 - biblio on intranet
ok 99 - biblio deleted
ok
All tests successful.
Files=1, Tests=99, 153 wallclock secs ( 0.05 usr  0.01 sys +  2.92 cusr  1.06 csys =  4.04 CPU)
Result: PASS

real    2m32.825s
user    0m3.048s
sys     0m1.096s
Comment 23 Mason James 2020-11-19 03:24:28 UTC
(In reply to Jonathan Druart from comment #19)
> Created attachment 113793 [details] [review] [review]
> Bug 26031: Remove www/search_utf8.t
> 
> Those tests are not reliable and fail randomly and we did not find the
> root of the problem.
> I am in favour of removing them, at least for now.

hi Jonathan, can you push my last patch to test on jenkins please
Comment 24 Jonathan Druart 2020-11-19 08:24:38 UTC
Can you explain your patch, Mason? Adding sleep everywhere in this script won't make the tests pass IMO.
The calls are synchronous, sleep(1) won't change anything. And even if it would, it's (almost) always a bad idea as 1s won't be enough in all cases (slow servers?)
Comment 25 Mason James 2020-11-19 13:12:19 UTC
Created attachment 113811 [details] [review]
Bug 26031: Remove 4 failing tests from t/db_dependent/www/search_utf8.t

to test...
1/ apply patch
2/ run test
3/ observe PASS

root@kohadevbox:/kohadevbox/koha# export KOHA_INTRANET_URL="http://kohadev-intra.myDNSname.org:8081"
root@kohadevbox:/kohadevbox/koha# export KOHA_OPAC_URL="http://kohadev.myDNSname.org:8080"
root@kohadevbox:/kohadevbox/koha# prove -v  ./t/db_dependent/www/search_utf8.t
1..87
ok 1 - connect to intranet
ok 2 - login to staff interface
ok 3 - load main page
ok 4 - open tools module
ok 5 - go to stage MARC
ok 6 - uploaded file
ok 7 - upload succeeded
ok 8 - reopen stage MARC page
ok 9 - stage MARC
ok 10 - have job ID
ok 11 - job was completed
ok 12 - reopen stage MARC page at end of upload
ok 13 - stage MARC
ok 14 - view batch
ok 15 - imported records into catalog
ok 16 - got search on intranet
ok 17 - got search on intranet
ok 18 - Base is like "(?^u:search.pl\?advsearch=1&idx=kw&q=%CE%91%CE%B8%CE%AE%CE%BD%CE%B1)"
ok 19 - UTF-8 are multi-byte. Good
ok 20 - UTF-8 chars are correctly present. Good
ok 21 - got opac
ok 22 - got opac
ok 23 - Base is like "(?^u:opac-search.pl\?(idx=&)?q=%CE%91%CE%B8%CE%AE%CE%BD%CE%B1)"
ok 24 - Base does not contain an idx
ok 25 - UTF-8 are multi-byte. Good
ok 26 - UTF-8 chars are correctly present. Good
ok 27 - view and clean batch
ok 28 - biblio on intranet
ok 29 - biblio deleted
ok 30 - connect to intranet
ok 31 - login to staff interface
ok 32 - load main page
ok 33 - open tools module
ok 34 - go to stage MARC
ok 35 - uploaded file
ok 36 - upload succeeded
ok 37 - reopen stage MARC page
ok 38 - stage MARC
ok 39 - have job ID
ok 40 - job was completed
ok 41 - reopen stage MARC page at end of upload
ok 42 - stage MARC
ok 43 - view batch
ok 44 - imported records into catalog
ok 45 - got search on intranet
ok 46 - got search on intranet
ok 47 - Base is like "(?^u:search.pl\?advsearch=1&idx=kw&q=Ram%C3%B2n)"
ok 48 - UTF-8 are multi-byte. Good
ok 49 - UTF-8 chars are correctly present. Good
ok 50 - got opac
ok 51 - got opac
ok 52 - Base is like "(?^u:opac-search.pl\?(idx=&)?q=Ram%C3%B2n)"
ok 53 - Base does not contain an idx
ok 54 - UTF-8 are multi-byte. Good
ok 55 - UTF-8 chars are correctly present. Good
ok 56 - view and clean batch
ok 57 - biblio on intranet
ok 58 - biblio deleted
ok 59 - connect to intranet
ok 60 - login to staff interface
ok 61 - load main page
ok 62 - open tools module
ok 63 - go to stage MARC
ok 64 - uploaded file
ok 65 - upload succeeded
ok 66 - reopen stage MARC page
ok 67 - stage MARC
ok 68 - have job ID
ok 69 - job was completed
ok 70 - reopen stage MARC page at end of upload
ok 71 - stage MARC
ok 72 - view batch
ok 73 - imported records into catalog
ok 74 - got search on intranet
ok 75 - got search on intranet
ok 76 - Base is like "(?^u:search.pl\?advsearch=1&idx=kw&q=%F0%A0%BB%BAtomasito%F0%A0%BB%BA)"
ok 77 - UTF-8 are multi-byte. Good
ok 78 - UTF-8 chars are correctly present. Good
ok 79 - got opac
ok 80 - got opac
ok 81 - Base is like "(?^u:opac-search.pl\?(idx=&)?q=%F0%A0%BB%BAtomasito%F0%A0%BB%BA)"
ok 82 - Base does not contain an idx
ok 83 - UTF-8 are multi-byte. Good
ok 84 - UTF-8 chars are correctly present. Good
ok 85 - view and clean batch
ok 86 - biblio on intranet
ok 87 - biblio deleted
ok
All tests successful.
Files=1, Tests=87, 127 wallclock secs ( 0.06 usr  0.02 sys +  3.00 cusr  0.99 csys =  4.07 CPU)
Result: PASS

real    2m6.709s
user    0m3.124s
sys     0m1.032s
Comment 26 Mason James 2020-11-19 13:15:22 UTC
(In reply to Jonathan Druart from comment #24)
> Can you explain your patch, Mason? Adding sleep everywhere in this script
> won't make the tests pass IMO.
> The calls are synchronous, sleep(1) won't change anything. And even if it
> would, it's (almost) always a bad idea as 1s won't be enough in all cases
> (slow servers?)

hi Jonathan, yes, i do see your point about slow servers,

i uploaded another patch that just removes the 4 unreliable tests
Comment 27 Jonathan Druart 2020-11-19 13:26:57 UTC
Created attachment 113812 [details] [review]
Bug 26031: Remove the failing tests and the diag
Comment 28 Jonathan Druart 2020-11-20 07:18:49 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 29 Mason James 2020-12-09 04:58:18 UTC
(In reply to Jonathan Druart from comment #28)
> Pushed to master for 20.11, thanks to everybody involved!

hi, can we get this pushed to 20.05 and 19.11
Comment 30 Andrew Fuerste-Henry 2020-12-09 21:23:57 UTC
Doesn't apply cleanly to 20.05, please rebase