Bug 12460

Summary: Search history: Combine to start a new search
Product: Koha Reporter: Julian Maurice <julian.maurice>
Component: SearchingAssignee: Julian Maurice <julian.maurice>
Status: Patch doesn't apply --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: aleisha, andreas.jonsson, anne-claire.bernaudin, bgkriegel, chris, claes.eriksson, claire.hernandez, gitbot, hagud, jonathan.druart+koha, luce.barbey, lucie.gay, nick, rochellehealy.student, tredok.pierre, veron
Version: masterKeywords: Academy
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27546
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 11430, 14699    
Bug Blocks:    
Attachments: Bug 12460: Allow to start a new search from multiple search history lines
Bug 12460: Allow to start a new search from multiple search history lines
Bug 12460: Allow to start a new search from multiple search history lines
Bug 12460: Allow to start a new search from multiple search history lines
Bug 12460: Allow to start a new search from multiple search history lines
Bug 12460: Allow to start a new search from multiple search history lines
Bug 12460: Simplify interface to combine queries
Bug 12460: Allow to start a new search from multiple search history lines
Bug 12460: Simplify interface to combine queries
Bug 12460: Fix query combination for anonymous users

Description Julian Maurice 2014-06-20 13:42:42 UTC
Allow to start a new search from multiple search history lines
Comment 1 Julian Maurice 2014-06-20 13:44:04 UTC Comment hidden (obsolete)
Comment 2 Bernardo Gonzalez Kriegel 2014-08-10 16:32:31 UTC
Trying to test, but Bug 11430 no longer applies
Comment 3 Aleisha Amohia 2014-10-28 04:25:12 UTC
Sent this error message when trying to test:
fatal: sha1 information is lacking or useless (C4/Search/History.pm).

Setting to 'patch does not apply'
Comment 4 Owen Leonard 2014-10-28 12:01:05 UTC
(In reply to Aleisha Amohia from comment #3)
> Sent this error message when trying to test:
> fatal: sha1 information is lacking or useless (C4/Search/History.pm).

Did you apply the patch for Bug 11430 first?
Comment 5 Rochelle Healy 2015-01-12 22:57:44 UTC
We tried applying bug 11430 first but it still doesn't apply, 

Applying: Bug 12460: Allow to start a new search from multiple search history lines
fatal: sha1 information is lacking or useless (koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-search-history.tt).
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.
Patch failed at 0001 Bug 12460: Allow to start a new search from multiple search history lines
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-12460-Allow-to-start-a-new-search-from-multipl-UuxtJ9.patch
Comment 6 Julian Maurice 2015-02-06 14:12:46 UTC Comment hidden (obsolete)
Comment 7 claire.hernandez@biblibre.com 2015-03-04 11:24:13 UTC
To add in test plan:
Enable syspref EnableSearchHistory before testing.
Comment 8 Pierre Angot 2015-03-04 14:26:05 UTC
I've tested it on sandbox.

It Works ! ^^
Comment 9 LuceBarbey 2015-03-04 16:33:23 UTC
I've tested it and I've some comments.
1.This function (developpement) needs a documentation about the behavior.
2. The first time we come on search history, we must select some lines and click on "start a new search" before to see the boolean operator 'NOT'. It's not intuitive
3. To build a complex search, we must proceed in several steps, because we could search only with AND (match all) / OR (match any.)
Proposals :
1. Display the "not" column immediately
2. Modify the form : when nothing is checked display "Select searches to run a combined search". When at least one search is checked, display "With selected searches run a combined search [matching any of selected searches] [ok]
Comment 10 Julian Maurice 2015-08-21 11:56:02 UTC Comment hidden (obsolete)
Comment 11 Julian Maurice 2015-08-21 12:03:30 UTC
Patch rebased on top of bug 14699

It also includes minor changes to address problem 2 of comment 9:
- "Start a new search" link is displayed on its own line and doesn't require that the user selected some searches to be "clickable"
- Add an alert() popup when user clicks on "Go" without having selected searches.

So, the expected user behavior is now:
1/ Click on "Start a new search". (The 'Not' column appears)
2/ Select some searches (optionally toggle the NOT option for some of these)
3/ Click on Go
Comment 12 Julian Maurice 2015-09-08 07:39:25 UTC
Created attachment 42450 [details] [review]
Bug 12460: Allow to start a new search from multiple search history lines

Rebased on top of bug 14699
Comment 13 I'm just a bot 2016-03-06 21:40:58 UTC
This bug depends on 14699 which is in status In Discussion
Comment 14 claire.hernandez@biblibre.com 2016-10-14 13:24:41 UTC
I am not a bot ;)
This bug depends on 14699 which is in status Needs Sign off.
Comment 15 Marc VĂ©ron 2016-12-16 08:32:59 UTC
Sorry, could not apply:

fatal: sha1 information is lacking or useless (koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/search-history.tt).
Comment 16 Julian Maurice 2017-01-20 17:12:18 UTC
Created attachment 59384 [details] [review]
Bug 12460: Allow to start a new search from multiple search history lines

This patch regroups searches of current and previous sessions to allow
selecting lines from both groups.
It adds a new column (session date) to allow to differentiate searches
of current session from other sessions.
Patch is for Intranet and OPAC (bootstrap only)

Test plan:
1/ Login to intranet interface
2/ Do some biblio searches
3/ Go to the search history page
4/ See you have a new action available "Start a new search", grayed
beacause you haven't selected any line
5/ Select two or more searches
6/ "Start a new search" is now enabled, click it.
7/ A dropdown list appears, asking you to chose between a "Match any"
search and a "Match all" search. There is also a new column in the table
("Not") allowing you to apply a NOT to the corresponding line.
8/ Make your choice, and then click on Go button, next to the dropdown
list. You will be redirected to the catalogue search results.
9/ Confirm that the results are correct.
10/ Set syspref opacthemes to 'bootstrap'
11/ Go to OPAC interface anonymously and repeat steps 2-9
12/ Login to OPAC interface and repeat steps 2-9
13/ Check that authority search history still works as before the patch
(except that there is only one table instead of two)
Comment 17 Julian Maurice 2017-01-20 17:12:48 UTC
Patch rebased on master
Comment 18 Aleisha Amohia 2017-01-25 00:25:37 UTC
I have a few issues with the patch:
1) There is a 'session date' column and a 'date' column. The 'date' column is filled, but every row in the 'session date' column says Unknown. This column should be removed.
2) The 'start a new search' line is clickable without selecting any rows, but will not execute the search until some rows are selected. The line should be greyed out initially, as stated in your test plan
3) My previous search history in both staff client and OPAC are gone.

Setting as Failed QA.
Comment 19 Claes 2017-02-16 15:07:42 UTC
Since this feature would appeal to more experienced user I think that it would be better to align with common search interfaces.

The option "Start a new search" is not intuitive at all. What we want to do is: "Combine sets/queries"

"Match any" and "match all" is not really common language and it is not intuitive. 

It seems that I must select either match any/ match all even if I just want to select line 4 NOT line 2. The results indicate that the search works as it should but it is certainly not clear for a user.

I have 2 suggestions: the first beeing a bit simpler and probably easier to apply. The second would be the most up to date solution.


My first suggestion for a design remake would be:
1) At the search history page add 2 new columns
2) One column with the heading "Delete Sets". In this column, each line/search set should have a tick box. Two buttons in the same heading with "select all" and the second button "X delete" 
3) The second column with the heading "Combine sets or Combine queries". In this heading two tick boxes: "AND" & "OR". And this column should also have a tick box for each line.
4) remove the big red button "Remove selected searches"
5) remove the first column with tick boxes

So how about the option "NOT"? My experience as information specialist is that this operator is not well understood and often misused by normal users. 


A second suggestion would complicate it for the developer :-) but here it comes:
1) The search history page should have a number for each search set/search querie. And the numbers should be in this format: #1, #2, #3
2) An extra column at the end of the line with tick boxes for each line and with the heading "Delete searches". As in 2) above.
3) A search line above the search history with the text Combine queries and a brief example (#1 OR #3) AND #6. As the example show there are no box ticking here but you send a querie using old search sets.
4) remove the big red button "Remove selected searches"
5) remove the first column with tick boxes
Comment 20 Julian Maurice 2018-02-22 15:55:16 UTC
Created attachment 72091 [details] [review]
Bug 12460: Allow to start a new search from multiple search history lines

This patch regroups searches of current and previous sessions to allow
selecting lines from both groups.
It adds a new column (session date) to allow to differentiate searches
of current session from other sessions.
Patch is for Intranet and OPAC

Test plan:
  1. Login to intranet interface
  2. Do some biblio searches
  3. Go to the search history page
  4. See you have a new action available "Start a new search", grayed
     because you haven't selected any line
  5. Select two or more searches
  6. "Start a new search" is now enabled, click it.
  7. A dropdown list appears, asking you to chose between a "Match any"
     search and a "Match all" search. There is also a new column in the
     table ("Not") allowing you to apply a NOT to the corresponding line
  8. Make your choice, and then click on Go button, next to the dropdown
     list. You will be redirected to the catalogue search results.
  9. Confirm that the results are correct.
 10. Set syspref opacthemes to 'bootstrap'
 11. Go to OPAC interface anonymously and repeat steps 2-9
 12. Login to OPAC interface and repeat steps 2-9
 13. Check that authority search history still works as before the patch
     (except that there is only one table instead of two)
Comment 21 Julian Maurice 2018-02-22 15:55:21 UTC
Created attachment 72092 [details] [review]
Bug 12460: Simplify interface to combine queries

The complicated interface using 'Match any/Match all', checkboxes and
'NOT' selectors is replaced by a simple textbox which expects queries
like this:

  (#1 or #3) and #6

where 1, 3 and 6 are IDs in search_history table.

A new column in the table show these ids.
Comment 22 Julian Maurice 2018-02-22 15:59:48 UTC
(In reply to Claes from comment #19) 
> A second suggestion would complicate it for the developer :-) but here it
> comes:
> 1) The search history page should have a number for each search set/search
> querie. And the numbers should be in this format: #1, #2, #3
> 2) An extra column at the end of the line with tick boxes for each line and
> with the heading "Delete searches". As in 2) above.
> 3) A search line above the search history with the text Combine queries and
> a brief example (#1 OR #3) AND #6. As the example show there are no box
> ticking here but you send a querie using old search sets.
> 4) remove the big red button "Remove selected searches"
> 5) remove the first column with tick boxes

I've chosen this 2nd option, as it is much more flexible, and was not that complicated to write (it even removes code!).
Comment 23 Andreas Jonsson 2018-03-08 18:29:44 UTC
The support for non-logged in users by getting the search history from the session was lost in the simplification patch (attachment 72092 [details] [review]).
Comment 24 Lucie Gay 2018-03-12 16:14:45 UTC
Some comments about my test :

  6. "Start a new search" is now enabled, click it.

-> "Start a new search" doesn't appear.

  7. A dropdown list appears, asking you to chose between a "Match any"
     search and a "Match all" search. There is also a new column in the
     table ("Not") allowing you to apply a NOT to the corresponding line
  8. Make your choice, and then click on Go button, next to the dropdown
     list. You will be redirected to the catalogue search results.

-> Instead, there is a simple text box and a button "Combine queries".

  9. Confirm that the results are correct.

-> Launch a new search with combination of IDs works well (for example : #1 AND #2) and generate a new line in the history search.

 10. Set syspref opacthemes to 'bootstrap'

-> During the test, bootstrap was already assigned to syspref opacthemes.

 11. Go to OPAC interface anonymously and repeat steps 2-9

-> When we are logged, the IDs of search are simple (#1), but when we are not logged the IDs looks like (#2018312170988)

 12. Login to OPAC interface and repeat steps 2-9
 13. Check that authority search history still works as before the patch
     (except that there is only one table instead of two)

-> There are still two tabs in the search history : catalog / authority. So it's not easy to combine both. In the authority's tab, there is no option "combine queries".
Comment 25 Julian Maurice 2018-03-12 16:24:30 UTC
Hi Lucie,

The feature has changed and the test plan was not updated. Sorry about that.
Here's an updated test plan:

Test plan:
  1. Login to intranet interface
  2. Do some biblio searches
  3. Go to the search history page
  4. See you have a new textbox below the table
  5. Type in the textbox a query using the IDs displayed in the table, like "(#1 or #2) and #3" and click on "Combine queries". You will be redirected to the catalogue search results.
  6. Confirm that the results are correct.
  7. Go to OPAC interface anonymously and repeat steps 2-6
  8. Login to OPAC interface and repeat steps 2-6
  9. Check that authority search history still works as before the patch
     (except that there is only one table instead of two)
Comment 26 Julian Maurice 2018-03-12 16:25:32 UTC
(In reply to Lucie Gay from comment #24)
> -> There are still two tabs in the search history : catalog / authority. So
> it's not easy to combine both. In the authority's tab, there is no option
> "combine queries".

That is expected, the feature is only for biblios.
Comment 27 Lucie Gay 2018-03-13 09:14:52 UTC
Thank you for the new test plan.
It's ok until step 7.
Anonymous mode : ID in first column looks like #20183139523615. If I combine two of these ID, it doesn't work
the request generated is 
ccl=allrecords,alwaysmatches='' AND allrecords,alwaysmatches=''
Comment 28 Hugo Agud 2018-03-13 09:31:25 UTC
I think the combination method is not quite intuitive for the end user, it is very powerful it gives you the chance  of complex combination but I guess, that perhaps user will prefer a more intuitive way of combine searches.

Once you are able to select the old searches, I think they will expect to be able just combine and that's all, in the same way.than mering bibliorecords.
Comment 29 Julian Maurice 2018-09-14 12:38:20 UTC
Created attachment 78648 [details] [review]
Bug 12460: Allow to start a new search from multiple search history lines

This patch regroups searches of current and previous sessions to allow
selecting lines from both groups.
It adds a new column (session date) to allow to differentiate searches
of current session from other sessions.
Patch is for Intranet and OPAC

Test plan:
  1. Login to intranet interface
  2. Do some biblio searches
  3. Go to the search history page
  4. See you have a new action available "Start a new search", grayed
     because you haven't selected any line
  5. Select two or more searches
  6. "Start a new search" is now enabled, click it.
  7. A dropdown list appears, asking you to chose between a "Match any"
     search and a "Match all" search. There is also a new column in the
     table ("Not") allowing you to apply a NOT to the corresponding line
  8. Make your choice, and then click on Go button, next to the dropdown
     list. You will be redirected to the catalogue search results.
  9. Confirm that the results are correct.
 10. Set syspref opacthemes to 'bootstrap'
 11. Go to OPAC interface anonymously and repeat steps 2-9
 12. Login to OPAC interface and repeat steps 2-9
 13. Check that authority search history still works as before the patch
     (except that there is only one table instead of two)
Comment 30 Julian Maurice 2018-09-14 12:38:26 UTC
Created attachment 78649 [details] [review]
Bug 12460: Simplify interface to combine queries

The complicated interface using 'Match any/Match all', checkboxes and
'NOT' selectors is replaced by a simple textbox which expects queries
like this:

  (#1 or #3) and #6

where 1, 3 and 6 are IDs in search_history table.

A new column in the table show these ids.
Comment 31 Julian Maurice 2018-09-14 12:38:31 UTC
Created attachment 78650 [details] [review]
Bug 12460: Fix query combination for anonymous users
Comment 32 Julian Maurice 2018-09-14 12:43:47 UTC
(In reply to Hugo Agud from comment #28)
> I think the combination method is not quite intuitive for the end user, it
> is very powerful it gives you the chance  of complex combination but I
> guess, that perhaps user will prefer a more intuitive way of combine
> searches.
> 
> Once you are able to select the old searches, I think they will expect to be
> able just combine and that's all, in the same way.than mering bibliorecords.
What do you mean exactly by "just combine" ?
Comment 33 Claes 2018-09-17 11:08:28 UTC
The test plan starts with:"Login to intranet interface" but this feature is about combining searches in the OPAC interface. 
Doing that in a ByWater sandbox everything in step 1-9 works fine with the modification concerning the simplified search box.

Instead of step 10 I logged out and repeated made searches as an anonymous. Then logged in in OPAC again and correctly found that these new searches were added to the old ones. 

Authority search works as before and is - as intended - without any possibility to combie searches.

However, I found two peculiar things, which is more about UX and not about how the bug works:
1. There are two date columns. "Session date" is set to unknown when you have left the active session. I can live with this since the column "date" has date and time for both old and active sessions.
2. Session ID is by default ordered in a descending order. Since you can rearrange by other columns, e.g. Search and Results, I looked for a way to resort in the Id column. It took me 10 seconds to find that the best way to sort in Id order is to use the Date column.
Comment 34 Jonathan Druart 2018-09-21 20:27:24 UTC
Hi Julian,
1. minor:
 FAIL   koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/search-history.tt
   FAIL   forbidden patterns
                forbidden pattern: Do not use line breaks inside template tags (bug 18675) (line 104)
                forbidden pattern: Do not use line breaks inside template tags (bug 18675) (line 114)

 FAIL   koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-search-history.tt
   FAIL   forbidden patterns
                forbidden pattern: Do not use line breaks inside template tags (bug 18675) (line 133)
                forbidden pattern: Do not use line breaks inside template tags (bug 18675) (line 144)

2. Bad news but... this will not work with Elastic, right?
Comment 35 Aleisha Amohia 2019-01-17 03:31:27 UTC
Patches need rebasing!

$ git bz apply 12460
Bug 12460 - Search history: Combine to start a new search

78648 - Bug 12460: Allow to start a new search from multiple search history lines
78649 - Bug 12460: Simplify interface to combine queries
78650 - Bug 12460: Fix query combination for anonymous users

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 12460: Allow to start a new search from multiple search history lines
Using index info to reconstruct a base tree...
M	koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/search-history.tt
M	koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-search-history.tt
Falling back to patching base and 3-way merge...
Auto-merging koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-search-history.tt
CONFLICT (content): Merge conflict in koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-search-history.tt
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/search-history.tt
CONFLICT (content): Merge conflict in koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/search-history.tt
Failed to merge in the changes.