Bug 20888

Summary: Allow use of boolean operator 'not' in item search
Product: Koha Reporter: Katrin Fischer <katrin.fischer>
Component: SearchingAssignee: Matthias Meusburger <matthias.meusburger>
Status: CLOSED FIXED QA Contact: Nick Clemens (kidclamp) <nick>
Severity: enhancement    
Priority: P5 - low CC: axelle.clarisse, fridolin.somers, matthias.meusburger, pierre-marc.thibault, sandboxes, tomascohen
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22595
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.11.00
Circulation function:
Bug Depends on: 22596    
Bug Blocks:    
Attachments: Bug 20888: Allow use of boolean operator 'not' in item search
Bug 20888: Allow use of boolean operator 'not' in item search
Bug 20888: Allow use of boolean operator 'not' in item search
Bug 20888: Allow use of boolean operator 'not' in item search
Bug 20888: Follow-up: 'is not' operator behaves as 'not like'
Bug 20888: Follow-up: revert unecessary change on jquery selector
Bug 20888: Allow use of boolean operator 'not' in item search
Bug 20888: Follow-up: 'is not' operator behaves as 'not like'
Bug 20888: Follow-up: revert unecessary change on jquery selector
Bug 20888: Allow use of boolean operator 'not' in item search
Bug 20888: (follow-up) 'is not' operator behaves as 'not like'
Bug 20888: (follow-up) revert unecessary change on jquery selector
Bug 20888: Allow use of boolean operator 'not' in item search
Bug 20888: (follow-up) 'is not' operator behaves as 'not like'
Bug 20888: (follow-up) revert unecessary change on jquery selector

Description Katrin Fischer 2018-06-06 16:21:15 UTC
Sometimes it's helpful to be able to exclude items from an item search, for example 'all callnumbers starting with 9'. The item search form currently offers 'or' and 'and', but 'not' is not an option.
Comment 1 Matthias Meusburger 2019-02-04 10:34:24 UTC
Created attachment 84692 [details] [review]
Bug 20888: Allow use of boolean operator 'not' in item search

Test plan:

 - Apply the patch
 - Check that you have a 'is not' option in the third box in item search
 - Check that you can exclude items from the search results with this 'is not' option
 - Check that you can still combine parameters with 'and' and 'or'
 - Check that the exclusion works for custom fields (see Administration > Item search fields)
 - Check that the exclusion works for custom fields linked to authorised values
Comment 2 Katrin Fischer 2019-02-04 11:42:46 UTC
Please don't forget to add yourself as assignee :)
Comment 3 Matthias Meusburger 2019-02-04 15:05:23 UTC
Oops, sorry :)
Comment 4 Pierre-Marc Thibault 2019-02-08 21:15:41 UTC
The patch does not pass the QA test :

FAIL	koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/itemsearch.tt
   FAIL	  filters
		missing_filter at line 237 (        var authorised_values = [% authorised_values_json %];)
   OK	  forbidden patterns
   OK	  git manipulation
   OK	  spelling
   OK	  tt_valid
   OK	  valid_template
Comment 5 Matthias Meusburger 2019-02-19 15:05:00 UTC
Created attachment 85281 [details] [review]
Bug 20888: Allow use of boolean operator 'not' in item search

Test plan:

 - Apply the patch
 - Check that you have a 'is not' option in the third box in item search
 - Check that you can exclude items from the search results with this 'is not' option
 - Check that you can still combine parameters with 'and' and 'or'
 - Check that the exclusion works for custom fields (see Administration > Item search fields)
 - Check that the exclusion works for custom fields linked to authorised values
Comment 6 Matthias Meusburger 2019-02-19 15:05:45 UTC
Fixed QA using $raw filter.
Comment 7 Fridolin Somers 2019-03-27 14:43:21 UTC
I have created a dedicated bug for raw filter.
Please rebase.
Comment 8 Katrin Fischer 2019-04-26 10:58:21 UTC
Matthias, can you please rebase? :)
Comment 9 Matthias Meusburger 2019-04-26 15:09:27 UTC
Frido, I don't get the status of this BZ or your comment.

The patch applies on master. What is to rebase exactly? Which bug are you referring to? What are you expecting me to do?
Comment 10 Fridolin Somers 2019-05-02 07:59:59 UTC
(In reply to Matthias Meusburger from comment #9)
> Frido, I don't get the status of this BZ or your comment.
> 
> The patch applies on master. What is to rebase exactly? Which bug are you
> referring to? What are you expecting me to do?

Sorry its really not clear (even for me).

I have created Bug 22596 for use of raw filter and added as dependency.

Now I see the raw filter already there in your patch so forget what I said ;)
Comment 11 Matthias Meusburger 2019-05-02 08:50:26 UTC
Thanks Frido.
Comment 12 Axelle Aix-Marseille Université 2019-10-01 08:10:38 UTC
We try to test today : this patch doesn't apply on a sandbox (Biblibre n°1)
Comment 13 Matthias Meusburger 2019-10-01 09:20:54 UTC
Created attachment 93347 [details] [review]
Bug 20888: Allow use of boolean operator 'not' in item search

Test plan:

 - Apply the patch
 - Check that you have a 'is not' option in the third box in item search
 - Check that you can exclude items from the search results with this 'is not' option
 - Check that you can still combine parameters with 'and' and 'or'
 - Check that the exclusion works for custom fields (see Administration > Item search fields)
 - Check that the exclusion works for custom fields linked to authorised values
Comment 14 Matthias Meusburger 2019-10-01 09:21:42 UTC
The patch has been rebased, it should apply correctly on the sandboxes now.
Comment 15 Biblibre Sandboxes 2019-10-01 13:54:12 UTC
Patch tested with a sandbox, by Amandine Zocca <azocca@ville-montauban.fr>
Comment 16 Biblibre Sandboxes 2019-10-01 13:54:43 UTC
Created attachment 93370 [details] [review]
Bug 20888: Allow use of boolean operator 'not' in item search

Test plan:

 - Apply the patch
 - Check that you have a 'is not' option in the third box in item search
 - Check that you can exclude items from the search results with this 'is not' option
 - Check that you can still combine parameters with 'and' and 'or'
 - Check that the exclusion works for custom fields (see Administration > Item search fields)
 - Check that the exclusion works for custom fields linked to authorised values

Signed-off-by: Amandine Zocca <azocca@ville-montauban.fr>
Comment 17 Katrin Fischer 2019-10-02 15:50:51 UTC
Wondering: If is translates to "like", should "not" not translate to "not like"? Currently it will be != which doesn't allow for truncation.

Example would be a search for: Title does not contain a certain keyword.
Comment 18 Matthias Meusburger 2019-10-03 07:40:54 UTC
I think we should rename "is" to "is like" then. This way it will be consistent, with the following options:

 - is like
 - is not

An "is" and/or "is not like" option may be added in the future, and it will still be consistent.

What do you think?
Comment 19 Katrin Fischer 2019-10-03 07:54:39 UTC
(In reply to Matthias Meusburger from comment #18)
> I think we should rename "is" to "is like" then. This way it will be
> consistent, with the following options:
> 
>  - is like
>  - is not
> 
> An "is" and/or "is not like" option may be added in the future, and it will
> still be consistent.
> 
> What do you think?

One problem we see with item search is that people expect it to work like the bibliographic search. For example, when searching for "Title is some words", they expect results to come up, but often forget to put the truncation. If the title actually is "these some words" the search will only find it if you use: Title is %some words".
I think that's the reason, why we should not change the wording, we do use the SQL operator "like" and unless you use truncation, it works like an exact search. But it gives you the option to use the truncation.

And I think we want the same behavior for adding the 'not' operator: If the libraries don't add truncation characters it will actually behave like !=, but they have the option to add them to make search a bit broader.

Also: The hint below the search options will then be accurate, otherwise it would be misleading:

You can use the following wildcard characters: % _
% matches any number of characters
_ matches only a single character
Comment 20 Matthias Meusburger 2019-10-03 07:56:51 UTC
Ok, understood. I'll submit a new patch.
Comment 21 Fridolin Somers 2019-11-12 09:26:45 UTC
Hi

-            $('div.form-field-select-text select[name="f"]').change(function() {
+            $('div.form-field-select-text select.form-field-column').change(function() {

is this change necessary ?
New select has name "op".
Comment 22 Matthias Meusburger 2020-01-20 14:46:49 UTC
Created attachment 97626 [details] [review]
Bug 20888: Follow-up: 'is not' operator behaves as 'not like'
Comment 23 Matthias Meusburger 2020-01-20 14:48:02 UTC
Katrin, here's the modification you asked for: 'is not' now behaves like 'not like' (just like 'is' behaves like 'like')
Comment 24 Matthias Meusburger 2020-01-20 14:48:30 UTC
Created attachment 97627 [details] [review]
Bug 20888: Follow-up: revert unecessary change on jquery selector
Comment 25 Matthias Meusburger 2020-01-20 14:49:15 UTC
Frido, you were right, this change was unnecessary and has been reverted.
Comment 26 Katrin Fischer 2020-01-21 07:07:32 UTC
Hi Matts, have been testing:

+        <select name="op" class="form-field-not">
+           <option value="like">is</option>
+            [% IF params.op == 'not like' %]
+               <option value="not like" selected="selected">is not</option>
+            [% ELSE %]
+               <option value="not like">is not</option>
+            [% END 

IF = not like... else = not like?

I tested with a custom item search option for "Withdrawn" linked to the WITHDRAWN authorised value. If I pick "is not" Withdrawn I get the only withdrawn item. If I pick 'is' withdrawn... I get all the other items. It works ok on master for the 'is' case.
Comment 27 Matthias Meusburger 2020-01-22 10:04:09 UTC
IF = not like... else = not like?

=> Yes, this is made for "not like" to be selected if needed when going back to the search. The "is" option is just above.

 
I tested with a custom item search option for "Withdrawn" linked to the WITHDRAWN authorised value. If I pick "is not" Withdrawn I get the only withdrawn item. If I pick 'is' withdrawn... I get all the other items. It works ok on master for the 'is' case.

=> I tested again, with the following configurations:

Unimarc : custom withdrawn search field => OK
Marc21  : custom withdrawn search field => OK
Marc21  : default withdrawn search field => OK

Could you possibly give it another go, Cait?
Comment 28 Katrin Fischer 2020-09-23 21:16:58 UTC
Hi Matts, could you please rebase? I am stuck with a:

error: could not build fake ancestor
Comment 29 Fridolin Somers 2020-10-06 07:49:11 UTC
Created attachment 111265 [details] [review]
Bug 20888: Allow use of boolean operator 'not' in item search

Test plan:

 - Apply the patch
 - Check that you have a 'is not' option in the third box in item search
 - Check that you can exclude items from the search results with this 'is not' option
 - Check that you can still combine parameters with 'and' and 'or'
 - Check that the exclusion works for custom fields (see Administration > Item search fields)
 - Check that the exclusion works for custom fields linked to authorised values

Signed-off-by: Amandine Zocca <azocca@ville-montauban.fr>
Comment 30 Fridolin Somers 2020-10-06 07:50:04 UTC
Created attachment 111266 [details] [review]
Bug 20888: Follow-up: 'is not' operator behaves as 'not like'
Comment 31 Fridolin Somers 2020-10-06 07:50:23 UTC
Created attachment 111267 [details] [review]
Bug 20888: Follow-up: revert unecessary change on jquery selector
Comment 32 Katrin Fischer 2020-10-25 02:18:32 UTC
Created attachment 112453 [details] [review]
Bug 20888: Allow use of boolean operator 'not' in item search

Test plan:

 - Apply the patch
 - Check that you have a 'is not' option in the third box in item search
 - Check that you can exclude items from the search results with this 'is not' option
 - Check that you can still combine parameters with 'and' and 'or'
 - Check that the exclusion works for custom fields (see Administration > Item search fields)
 - Check that the exclusion works for custom fields linked to authorised values

Signed-off-by: Amandine Zocca <azocca@ville-montauban.fr>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 33 Katrin Fischer 2020-10-25 02:18:38 UTC
Created attachment 112454 [details] [review]
Bug 20888: (follow-up) 'is not' operator behaves as 'not like'

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 34 Katrin Fischer 2020-10-25 02:18:44 UTC
Created attachment 112455 [details] [review]
Bug 20888: (follow-up) revert unecessary change on jquery selector

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 35 Nick Clemens (kidclamp) 2020-10-30 12:31:54 UTC
Created attachment 112691 [details] [review]
Bug 20888: Allow use of boolean operator 'not' in item search

Test plan:

 - Apply the patch
 - Check that you have a 'is not' option in the third box in item search
 - Check that you can exclude items from the search results with this 'is not' option
 - Check that you can still combine parameters with 'and' and 'or'
 - Check that the exclusion works for custom fields (see Administration > Item search fields)
 - Check that the exclusion works for custom fields linked to authorised values

Signed-off-by: Amandine Zocca <azocca@ville-montauban.fr>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 36 Nick Clemens (kidclamp) 2020-10-30 12:31:58 UTC
Created attachment 112692 [details] [review]
Bug 20888: (follow-up) 'is not' operator behaves as 'not like'

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 37 Nick Clemens (kidclamp) 2020-10-30 12:32:01 UTC
Created attachment 112693 [details] [review]
Bug 20888: (follow-up) revert unecessary change on jquery selector

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 38 Jonathan Druart 2020-11-04 12:02:39 UTC
Pushed to master for 20.11, thanks to everybody involved!