Bug 37430

Summary: (Bug 33407 follow-up) ISBD punctuation removal in ES searches
Product: Koha Reporter: Janusz Kaczmarek <januszop>
Component: Searching - ElasticsearchAssignee: Janusz Kaczmarek <januszop>
Status: Pushed to main --- QA Contact: Nick Clemens (kidclamp) <nick>
Severity: trivial    
Priority: P5 - low CC: martin.renvoize, nick
Version: unspecified   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
24.11.00
Circulation function:
Bug Depends on: 33407    
Bug Blocks:    
Attachments: Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches
Bug 37430: (Bug 33407 follow-up) Unit tests
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches
Bug 37430: (Bug 33407 follow-up) Unit tests
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches
Bug 37430: (Bug 33407 follow-up) Unit tests

Description Janusz Kaczmarek 2024-07-23 10:13:44 UTC
Tiny adjustment for very special and malicious cases needed in the Koha::SearchEngine::Elasticsearch::QueryBuilder::clean_search_term. 

Extended unit tests should be added following Nick's suggestions in Bug 33407 discussion.
Comment 1 Janusz Kaczmarek 2024-07-23 11:32:00 UTC
Created attachment 169408 [details] [review]
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches

Tiny adjustment for very special and malicious cases needed in
Koha::SearchEngine::Elasticsearch::QueryBuilder::clean_search_term.

Extended unit tests added following Nick's suggestions in
Bug 33407 discussion.

Test plan
=========
1. Use ktd with Elasticsearch and ktd's test data.
2. Edit the 4th record (The C programming language) and add a semicolon
   followed by a space before the slash in 245 $a to have:
   The C programming language ; /
3. Make a search for the first ISBD zone taken verbatim from the forth
   record:
   The C programming language ; / Brian W. Kernighan, Dennis M. Ritchie

   There should be no result.
4. Apply the patch, restart plack.
5. Repeat the search. You should get the record.

Sponsored-by: Ignatianum University in Cracow
Comment 2 Janusz Kaczmarek 2024-07-23 11:32:03 UTC
Created attachment 169409 [details] [review]
Bug 37430: (Bug 33407 follow-up) Unit tests
Comment 3 Roman Dolny 2024-07-23 14:18:45 UTC
Created attachment 169423 [details] [review]
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches

Tiny adjustment for very special and malicious cases needed in
Koha::SearchEngine::Elasticsearch::QueryBuilder::clean_search_term.

Extended unit tests added following Nick's suggestions in
Bug 33407 discussion.

Test plan
=========
1. Use ktd with Elasticsearch and ktd's test data.
2. Edit the 4th record (The C programming language) and add a semicolon
   followed by a space before the slash in 245 $a to have:
   The C programming language ; /
3. Make a search for the first ISBD zone taken verbatim from the forth
   record:
   The C programming language ; / Brian W. Kernighan, Dennis M. Ritchie

   There should be no result.
4. Apply the patch, restart plack.
5. Repeat the search. You should get the record.

Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Comment 4 Roman Dolny 2024-07-23 14:18:47 UTC
Created attachment 169424 [details] [review]
Bug 37430: (Bug 33407 follow-up) Unit tests

Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Comment 5 Nick Clemens (kidclamp) 2024-09-17 11:14:57 UTC
Created attachment 171606 [details] [review]
Bug 37430: (Bug 33407 follow-up) ISBD punctuation removal in ES searches

Tiny adjustment for very special and malicious cases needed in
Koha::SearchEngine::Elasticsearch::QueryBuilder::clean_search_term.

Extended unit tests added following Nick's suggestions in
Bug 33407 discussion.

Test plan
=========
1. Use ktd with Elasticsearch and ktd's test data.
2. Edit the 4th record (The C programming language) and add a semicolon
   followed by a space before the slash in 245 $a to have:
   The C programming language ; /
3. Make a search for the first ISBD zone taken verbatim from the forth
   record:
   The C programming language ; / Brian W. Kernighan, Dennis M. Ritchie

   There should be no result.
4. Apply the patch, restart plack.
5. Repeat the search. You should get the record.

Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 6 Nick Clemens (kidclamp) 2024-09-17 11:14:59 UTC
Created attachment 171607 [details] [review]
Bug 37430: (Bug 33407 follow-up) Unit tests

Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 7 Martin Renvoize (ashimema) 2024-10-02 09:56:12 UTC
Thanks for all the hard work!

Pushed to main for the next 24.11.00 release as RM Assistant