Bug 28603

Summary: check-url-quick.pl does not encode spaces in urls
Product: Koha Reporter: Nick Clemens <nick>
Component: Command-line UtilitiesAssignee: Nick Clemens <nick>
Status: Failed QA --- QA Contact: Marcel de Rooy <m.de.rooy>
Severity: normal    
Priority: P5 - low CC: m.de.rooy, marjorie.barry-vila, robin
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 28603: Encode spaces in urls in url checker
Bug 28603: Encode spaces in urls in url checker

Description Nick Clemens 2021-06-18 19:38:01 UTC
To recreate:
1 - Add an 856$u to a record with:
    http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street shuffle
2 - perl misc/cronjobs/check-url-quick.pl > test.txt 2>&1
3 - grep shuffle test.txt
4 - 303	http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street shuffle	596 Connection timed out
          'URL' => 'http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street shuffle'
303	http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street shuffle	596 Connection timed out
Comment 1 Nick Clemens 2021-06-18 19:43:18 UTC
Created attachment 122177 [details] [review]
Bug 28603: Encode spaces in urls in url checker

To test:
1 - Add a url to a record like:
    http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street%20shuffle
2 - perl misc/cronjobs/check-url-quick.pl -v > test.txt
3 - grep shuffle test.txt
4 - 303	http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street shuffle	596 Connection timed out
5 - Apply patch
6 - perl misc/cronjobs/check-url-quick.pl -v > test.txt
7 - grep shuffle test.txt
8 - 303	http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street%20shuffle	200 OK
Comment 2 David Nind 2021-06-18 20:35:11 UTC
Created attachment 122182 [details] [review]
Bug 28603: Encode spaces in urls in url checker

To test:
1 - Add a url to a record like:
    http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street%20shuffle
2 - perl misc/cronjobs/check-url-quick.pl -v > test.txt
3 - grep shuffle test.txt
4 - 303	http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street shuffle	596 Connection timed out
5 - Apply patch
6 - perl misc/cronjobs/check-url-quick.pl -v > test.txt
7 - grep shuffle test.txt
8 - 303	http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=ti&q=street%20shuffle	200 OK

Signed-off-by: David Nind <david@davidnind.com>
Comment 3 Marcel de Rooy 2021-06-23 14:47:25 UTC
                         $url = encode_utf8($url);
+                        $url = uri_escape( $url, " " );

Normally we escape with uri_escape_utf8. This appears to be not exactly the same. Please explain why we should do so?
Comment 4 Marcel de Rooy 2021-06-23 14:48:12 UTC
uri_escape_utf8( $string, $unsafe )
Works like uri_escape(), but will encode chars as UTF-8 before escaping them. This makes this function able to deal with characters with code above 255 in $string. Note that chars in the 128 .. 255 range will be escaped differently by this function compared to what uri_escape() would.