Bug 32406 - Cannot search pending orders using non-latin-1 scripts
Summary: Cannot search pending orders using non-latin-1 scripts
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Acquisitions (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: David Cook
QA Contact: Testopia
URL:
Keywords:
Depends on: 30165
Blocks: 32401
  Show dependency treegraph
 
Reported: 2022-12-05 22:47 UTC by David Cook
Modified: 2023-12-28 20:44 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.05.00,22.11.03,22.05.10,21.11.16
Circulation function:


Attachments
Bug 32406: Switch acq datatable from headers to query param (1.79 KB, patch)
2022-12-05 22:58 UTC, David Cook
Details | Diff | Splinter Review
Bug 32406: Switch acq datatable from headers to query param (1.89 KB, patch)
2022-12-06 13:26 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32406: Switch acq datatable from headers to query param (1.96 KB, patch)
2023-02-01 10:09 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description David Cook 2022-12-05 22:47:39 UTC
Due to the flaw with x-koha-query mentioned in bug 32401, it's not possible to search pending orders in Acquisitions using characters outside the Latin-1 character set.
Comment 1 David Cook 2022-12-05 22:49:07 UTC
The solution to this problem is to use the "q" parameter instead of the "x-koha-query" header.

While the "q" parameter runs the risk of creating too long of a URL, it is more functional than the "x-koha-query" header. 

Really we should be doing POSTs to the REST API when using DataTables, but the "good enough" fix here is to just change "header_filter" from "true" to "false" in koha-tmpl/intranet-tmpl/prog/en/modules/acqui/parcel.tt
Comment 2 David Cook 2022-12-05 22:58:25 UTC
Created attachment 144432 [details] [review]
Bug 32406: Switch acq datatable from headers to query param

This patch changes the pending orders datatable from using the
x-koha-query header to the "q" query parameter, so that it is
able to support UTF-8 encoded scripts rather than just Latin-1
scripts.

Test plan:

1. Apply patch
2. Go to
http://localhost:8081/cgi-bin/koha/cataloguing/addbiblio.pl?biblionumber=29
3. Change title from "Gairm" to "五" (alternatively use "cœur" or Arabic title)
4. Create a basket containing multiple records including bib 29 五
5. Close the basket
6. Receive shipment
7. You should be on a page like the following:
http://localhost:8081/cgi-bin/koha/acqui/parcel.pl?invoiceid=3
8. In the search box, type in 五
9. Note that you see a "Processing..." box and then the
results are filtered to just show the order for 五
Comment 3 Jonathan Druart 2022-12-06 09:18:57 UTC
Shouldn't we simply remove the line? :) false is the default value.
Comment 4 Tomás Cohen Arazi (tcohen) 2022-12-06 13:26:05 UTC
Created attachment 144446 [details] [review]
Bug 32406: Switch acq datatable from headers to query param

This patch changes the pending orders datatable from using the
x-koha-query header to the "q" query parameter, so that it is
able to support UTF-8 encoded scripts rather than just Latin-1
scripts.

Test plan:

1. Apply patch
2. Go to
http://localhost:8081/cgi-bin/koha/cataloguing/addbiblio.pl?biblionumber=29
3. Change title from "Gairm" to "五" (alternatively use "cœur" or Arabic title)
4. Create a basket containing multiple records including bib 29 五
5. Close the basket
6. Receive shipment
7. You should be on a page like the following:
http://localhost:8081/cgi-bin/koha/acqui/parcel.pl?invoiceid=3
8. In the search box, type in 五
9. Note that you see a "Processing..." box and then the
results are filtered to just show the order for 五

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
tcohen amend: removed the line instead as we are probably phasing out
that DT configuration option anyway.
Comment 5 Marcel de Rooy 2022-12-09 10:12:46 UTC
Just posted a comment on 32401 about encoding header. This is a move from header to URL parameter.
Could we have a privacy issue with exposing search terms like that, since URLs are much more visible? Log files, etc.
Comment 6 David Cook 2022-12-12 05:12:16 UTC
(In reply to Marcel de Rooy from comment #5)
> Just posted a comment on 32401 about encoding header. This is a move from
> header to URL parameter.
> Could we have a privacy issue with exposing search terms like that, since
> URLs are much more visible? Log files, etc.

For bug 32406, I wouldn't worry about it, because it's order data. It's unlikely to be sensitive.

But it's an interesting point. In master, the search is already put in the "q" parameter for http://localhost:8081/cgi-bin/koha/members/member.pl

I suppose things like email addresses, phone numbers, usernames, etc could potentially be passed through that and wind up in logs. It wouldn't necessarily be that coherent though.

But I think it would be fair to argue that searches should be POSTs. 

Of course, that gets complicated in terms of a RESTful interface. 

I think one could consider the "search" to be the resource though, so POSTing to create a search makes sense. 

Bit of a philosophical-technical discussion to have there I think.
Comment 7 David Cook 2022-12-12 05:17:30 UTC
But in the context of bug 32406, I think it would be fair to push QA, as it would be making the usage consistent across Koha.
Comment 8 Jonathan Druart 2023-02-01 10:09:59 UTC
Created attachment 145924 [details] [review]
Bug 32406: Switch acq datatable from headers to query param

This patch changes the pending orders datatable from using the
x-koha-query header to the "q" query parameter, so that it is
able to support UTF-8 encoded scripts rather than just Latin-1
scripts.

Test plan:

1. Apply patch
2. Go to
http://localhost:8081/cgi-bin/koha/cataloguing/addbiblio.pl?biblionumber=29
3. Change title from "Gairm" to "五" (alternatively use "cœur" or Arabic title)
4. Create a basket containing multiple records including bib 29 五
5. Close the basket
6. Receive shipment
7. You should be on a page like the following:
http://localhost:8081/cgi-bin/koha/acqui/parcel.pl?invoiceid=3
8. In the search box, type in 五
9. Note that you see a "Processing..." box and then the
results are filtered to just show the order for 五

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
tcohen amend: removed the line instead as we are probably phasing out
that DT configuration option anyway.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 9 Tomás Cohen Arazi (tcohen) 2023-02-02 15:01:59 UTC
Pushed to master for 23.05.

Nice work everyone, thanks!
Comment 10 Jacob O'Mara 2023-02-06 15:58:12 UTC
Nice work, thanks everyone!

Pushed to 22.11.x for the next release.
Comment 11 Lucas Gass (lukeg) 2023-02-14 21:33:04 UTC
Backported to 22.05.x for upcoming 22.05.10
Comment 12 Arthur Suzuki 2023-02-17 10:00:46 UTC
applied to 21.11.x for 21.11.16
Comment 13 Wainui Witika-Park 2023-03-15 01:43:34 UTC
Not backported to 21.05.x