Bug 37430 - (Bug 33407 follow-up) ISBD punctuation removal in ES searches
Summary: (Bug 33407 follow-up) ISBD punctuation removal in ES searches
Status: Pushed to main
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low trivial
Assignee: Janusz Kaczmarek
QA Contact: Nick Clemens (kidclamp)
URL:
Keywords:
Depends on: 33407
Blocks:
  Show dependency treegraph
 
Reported: 2024-07-23 10:13 UTC by Janusz Kaczmarek
Modified: 2024-10-02 09:56 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
24.11.00
Circulation function:


Attachments
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches (2.33 KB, patch)
2024-07-23 11:32 UTC, Janusz Kaczmarek
Details | Diff | Splinter Review
Bug 37430: (Bug 33407 follow-up) Unit tests (4.75 KB, patch)
2024-07-23 11:32 UTC, Janusz Kaczmarek
Details | Diff | Splinter Review
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches (2.39 KB, patch)
2024-07-23 14:18 UTC, Roman Dolny
Details | Diff | Splinter Review
Bug 37430: (Bug 33407 follow-up) Unit tests (4.81 KB, patch)
2024-07-23 14:18 UTC, Roman Dolny
Details | Diff | Splinter Review
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches (2.44 KB, patch)
2024-09-17 11:14 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 37430: (Bug 33407 follow-up) Unit tests (4.86 KB, patch)
2024-09-17 11:14 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Janusz Kaczmarek 2024-07-23 10:13:44 UTC
Tiny adjustment for very special and malicious cases needed in the Koha::SearchEngine::Elasticsearch::QueryBuilder::clean_search_term. 

Extended unit tests should be added following Nick's suggestions in Bug 33407 discussion.
Comment 1 Janusz Kaczmarek 2024-07-23 11:32:00 UTC
Created attachment 169408 [details] [review]
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches

Tiny adjustment for very special and malicious cases needed in
Koha::SearchEngine::Elasticsearch::QueryBuilder::clean_search_term.

Extended unit tests added following Nick's suggestions in
Bug 33407 discussion.

Test plan
=========
1. Use ktd with Elasticsearch and ktd's test data.
2. Edit the 4th record (The C programming language) and add a semicolon
   followed by a space before the slash in 245 $a to have:
   The C programming language ; /
3. Make a search for the first ISBD zone taken verbatim from the forth
   record:
   The C programming language ; / Brian W. Kernighan, Dennis M. Ritchie

   There should be no result.
4. Apply the patch, restart plack.
5. Repeat the search. You should get the record.

Sponsored-by: Ignatianum University in Cracow
Comment 2 Janusz Kaczmarek 2024-07-23 11:32:03 UTC
Created attachment 169409 [details] [review]
Bug 37430: (Bug 33407 follow-up) Unit tests
Comment 3 Roman Dolny 2024-07-23 14:18:45 UTC
Created attachment 169423 [details] [review]
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches

Tiny adjustment for very special and malicious cases needed in
Koha::SearchEngine::Elasticsearch::QueryBuilder::clean_search_term.

Extended unit tests added following Nick's suggestions in
Bug 33407 discussion.

Test plan
=========
1. Use ktd with Elasticsearch and ktd's test data.
2. Edit the 4th record (The C programming language) and add a semicolon
   followed by a space before the slash in 245 $a to have:
   The C programming language ; /
3. Make a search for the first ISBD zone taken verbatim from the forth
   record:
   The C programming language ; / Brian W. Kernighan, Dennis M. Ritchie

   There should be no result.
4. Apply the patch, restart plack.
5. Repeat the search. You should get the record.

Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Comment 4 Roman Dolny 2024-07-23 14:18:47 UTC
Created attachment 169424 [details] [review]
Bug 37430: (Bug 33407 follow-up) Unit tests

Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Comment 5 Nick Clemens (kidclamp) 2024-09-17 11:14:57 UTC
Created attachment 171606 [details] [review]
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches

Tiny adjustment for very special and malicious cases needed in
Koha::SearchEngine::Elasticsearch::QueryBuilder::clean_search_term.

Extended unit tests added following Nick's suggestions in
Bug 33407 discussion.

Test plan
=========
1. Use ktd with Elasticsearch and ktd's test data.
2. Edit the 4th record (The C programming language) and add a semicolon
   followed by a space before the slash in 245 $a to have:
   The C programming language ; /
3. Make a search for the first ISBD zone taken verbatim from the forth
   record:
   The C programming language ; / Brian W. Kernighan, Dennis M. Ritchie

   There should be no result.
4. Apply the patch, restart plack.
5. Repeat the search. You should get the record.

Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 6 Nick Clemens (kidclamp) 2024-09-17 11:14:59 UTC
Created attachment 171607 [details] [review]
Bug 37430: (Bug 33407 follow-up) Unit tests

Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 7 Martin Renvoize (ashimema) 2024-10-02 09:56:12 UTC
Thanks for all the hard work!

Pushed to main for the next 24.11.00 release as RM Assistant