Bug 26578 - OverDrive results can return false positives when searches contain CCL syntax
Summary: OverDrive results can return false positives when searches contain CCL syntax
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: OPAC (show other bugs)
Version: master
Hardware: All All
: P5 - low normal (vote)
Assignee: Nick Clemens
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-09-30 12:57 UTC by Nick Clemens
Modified: 2021-06-14 21:35 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Text to go in the release notes:
Version(s) released in:
21.05.00,20.11.03


Attachments
Bug 26578: Clean overdrive searches before passing to OD (2.22 KB, patch)
2020-09-30 13:03 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26578: Clean overdrive searches before passing to OD (3.45 KB, patch)
2020-11-23 13:21 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26578: Clean overdrive searches before passing to OD (3.45 KB, patch)
2020-11-30 14:28 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26578: Clean overdrive searches before passing to OD (3.51 KB, patch)
2020-12-02 14:38 UTC, Kelly McElligott
Details | Diff | Splinter Review
Bug 26578: Clean overdrive searches before passing to OD (3.61 KB, patch)
2021-01-29 10:21 UTC, Marcel de Rooy
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens 2020-09-30 12:57:31 UTC
This problem is from the ':' and '=' signs, along with field codes, that can be passed to overdrive when forming queries or clicking links. Depending on the overdrive catalog, these can match records causing inaccurate results

To recreate:
1 - Have OverDrive enabled and configured
2 - set UseAuthoritiesForTracings to "Don't use"
3 - Search in the catalog for a record
4 - From the details page for a record click on an author link
5 - Note the query in catalog is like 'au:"Whitford, Bradley."'
6 - Some overdrive catalogs will return records matching 'au:'
Comment 1 Nick Clemens 2020-09-30 13:03:34 UTC Comment hidden (obsolete)
Comment 2 Nick Clemens 2020-11-23 13:21:16 UTC Comment hidden (obsolete)
Comment 3 Nick Clemens 2020-11-30 14:28:33 UTC
Created attachment 114058 [details] [review]
Bug 26578: Clean overdrive searches before passing to OD

This patch copies regex used in buildQuery to detect ccl code in queries, but here we adapt
it to remove any ccl signifiers to ensure correct results from OD

To test:
1 - Have OverDrive enabled and configured
2 - set UseAuthoritiesForTracings to "Don't use"
3 - Search in the catalog for a record
4 - From the details page for a record click on an author link
5 - Note the query in catalog is like 'au:"Whitford, Bradley."'
6 - Some overdrive catalogs will return records matching 'au:'
7 - Apply patch
8 - Reload the page, if there were spurious results before, they should now be gone
9 - Perform a search that returns overdrive results
10 - For a title in overdrive search koha with:
    au:{Author} AND ti:{Title}
11 - Confirm overdrive results are returned
12 - Check the link to overdrive results and note query section is:
    q=Author AND Title


Current status: Needs Signoff
Comment 4 Kelly McElligott 2020-12-02 14:38:03 UTC
Created attachment 114098 [details] [review]
Bug 26578: Clean overdrive searches before passing to OD

This patch copies regex used in buildQuery to detect ccl code in queries, but here we adapt
it to remove any ccl signifiers to ensure correct results from OD

To test:
1 - Have OverDrive enabled and configured
2 - set UseAuthoritiesForTracings to "Don't use"
3 - Search in the catalog for a record
4 - From the details page for a record click on an author link
5 - Note the query in catalog is like 'au:"Whitford, Bradley."'
6 - Some overdrive catalogs will return records matching 'au:'
7 - Apply patch
8 - Reload the page, if there were spurious results before, they should now be gone
9 - Perform a search that returns overdrive results
10 - For a title in overdrive search koha with:
    au:{Author} AND ti:{Title}
11 - Confirm overdrive results are returned
12 - Check the link to overdrive results and note query section is:
    q=Author AND Title

Signed-off-by: Kelly McElligott <kelly@bywatersolutions.com>
Comment 5 Marcel de Rooy 2021-01-29 10:21:32 UTC
Created attachment 116014 [details] [review]
Bug 26578: Clean overdrive searches before passing to OD

This patch copies regex used in buildQuery to detect ccl code in queries, but here we adapt
it to remove any ccl signifiers to ensure correct results from OD

To test:
1 - Have OverDrive enabled and configured
2 - set UseAuthoritiesForTracings to "Don't use"
3 - Search in the catalog for a record
4 - From the details page for a record click on an author link
5 - Note the query in catalog is like 'au:&quot;Whitford, Bradley.&quot;'
6 - Some overdrive catalogs will return records matching 'au:'
7 - Apply patch
8 - Reload the page, if there were spurious results before, they should now be gone
9 - Perform a search that returns overdrive results
10 - For a title in overdrive search koha with:
    au:{Author} AND ti:{Title}
11 - Confirm overdrive results are returned
12 - Check the link to overdrive results and note query section is:
    q=Author AND Title

Signed-off-by: Kelly McElligott <kelly@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 6 Marcel de Rooy 2021-01-29 10:23:50 UTC
Looks good to me.
Cannot actually test OverDrive.
Decoding html entities with DomParser is a recommended approach.
This change will only affect people with OD enabled. No harm expected.
Comment 7 Jonathan Druart 2021-02-01 16:26:28 UTC
Pushed to master for 21.05, thanks to everybody involved!
Comment 8 Fridolin Somers 2021-02-05 15:46:11 UTC
Pushed to 20.11.x for 20.11.03
Comment 9 Andrew Fuerste-Henry 2021-02-09 12:56:25 UTC
Doesn't apply cleanly to 20.05, please rebase if needed