Created attachment 114168 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he starting search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! We need do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). If need do 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 4. For add information in the index we need run script for indexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find "AutocompleteElasticSearch". Value must be "Show". 7. Now we have Autocomplete for all search input (advanced search also)
Created attachment 114169 [details] Result
Created attachment 114945 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he starting search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! We need do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). If need do 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 4. For add information in the index we need run script for indexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find "AutocompleteElasticSearch". Value must be "Show". 7. Now we have Autocomplete for all search input (advanced search also) Bug 27113: ElasticSearch: Autocomplete in input search v.2.0 Update v.2 - API passing tests: prove t - fixed bugs
Whaou I like this :D Just a question : The patch changes some unrelated text in installer/data/mysql/mandatory/sysprefs.sql : -('AnonymousPatron','0',NULL,'Set the identifier (borrowernumber) of the anonymous patron. Used for suggestion and checkout history privacy',''), +('AnonymousPatron','0',NULL,'Set the identifier (borrowernumber) of the anonymous patron. Used for Suggestion and reading history privacy',''), Looks like is it unwanted.
In koha-tmpl/opac-tmpl/bootstrap/en/includes/opac-bottom.inc + var OL_PREVIEW = _("Preview"); Why this ?
I see : api/elasticsearch/intranet-autocomplete.pl This is strange, its not using the REST API right ? Its more a perl script like : opac/svc/elasticsearch/opac-autocomplete.pl Should be in svc/elasticsearch I'd say. BTW : It is possible to use the REST API ? It whould provide the service for both opac and intranet. Best regards,
Yes, I see many errors in the files. I will change it. I would like to provide the service for both opac and intranet through REST API. But, I see many preferences for REST API: ('RESTBasicAuth'- 'If enabled, Basic authentication is enabled for the REST API.','YesNo'), ('RESTdefaultPageSize' - 'Default page size for endpoints listing objects','Integer'), ('RESTOAuth2ClientCredentials' - 'If enabled, the OAuth2 client credentials flow is enabled for the REST API.','YesNo'), ('RESTPublicAnonymousRequests' -'If enabled, the API will allow anonymous access to public routes that don\'t require authenticated access.','YesNo'), ('RESTPublicAPI' - 'If enabled, the REST API will expose the /public endpoints.','YesNo') I am not sure that good idea for OPAC because REST API uses Auth. It is possible to disable authentication in the preferences for REST API, but this may have a consequences... Do you have a solution about this? Thanks!
I think you can provide a public API for OPAC : https://wiki.koha-community.org/wiki/REST_api_RFCs#Unprivileged_.28public.29_routes datas are the same as searching with OPAC website.
Are you sure that OPAC search uses REST API? It's only GET request (with parametres 'q', 'idx', ...) for file opac-search.pl and return a results for template *-results.tt. I don't see here REST API. Best regards,
Created attachment 115117 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he starting search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! We need do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). If need do 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 4. For add information in the index we need run script for indexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find "AutocompleteElasticSearch". Value must be "Show". 7. Now we have Autocomplete for all search input (advanced search also) Bug 27113: ElasticSearch: Autocomplete in input search v.2.0 Bug 27113: ElasticSearch: Autocomplete on input search. Code change
(In reply to Ivan Dziuba from comment #9) > Are you sure that OPAC search uses REST API? It's only GET request (with > parametres 'q', 'idx', ...) for file opac-search.pl and return a results for > template *-results.tt. > I don't see here REST API. > > Best regards, No indeed, search is not actually using API. I say your autocomplete feature should use a dedicated API service.
Created attachment 116293 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Bug 27113: ElasticSearch: Autocomplete in input search. Fix bugs.
Created attachment 119979 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Bug 27113: ElasticSearch: Autocomplete in input search. Fix bugs. Bug 27113: ElasticSearch: Autocomplete in input search. Use Unicode::Normalize instead Text::Unaccent.
This is very cool, and is working well in the OPAC. I don't think you need to add a separate CSS file. I think it should be incorporated into the main one. It's not a large enough addition to necessitate a separate file. I wonder if we should have an entry at the end of the autocomplete menu every time which the user can click on to submit the string they've entered? For instance, I type "dogs," and the autocomplete shows the first 10 results. A last menu item could say, "See full search results for <dogs>" or something like that? This patch adds the feature to the OPAC and staff interface but only adds one preference. I think it should be separated into two, one for each interface. It doesn't work for me in the staff interface. There is an error: "Could not compile /kohadevbox/koha/api/elasticsearch/autocomplete.pl: Can't locate Switch.pm in @INC (you may need to install the Switch module)" There are many errors flagged by the QA tool. There are instances of "console.log" in the code which must be removed.
Good idea on the button. But I asked Ivan to focus on feature at hand and everything else can be a new BZ. If he gets it through, it'll be easy to add on afterward. For the .css, you mean merging autocomplete.css into opac.css ? Couldn't that be a RM thing at the end, if really needed? Otherwise that makes the rebasing very painful. Ivan is certainly not at ease with that demand.
(In reply to Blou from comment #15) > For the .css, you mean merging autocomplete.css into opac.css ? I mean adding the contents of autocomplete.css into opac.scss. I'd be happy to submit a follow-up for this part, converting the CSS to SCSS, if that would help.
(In reply to Blou from comment #15) > Good idea on the button. But I asked Ivan to focus on feature at hand and > everything else can be a new BZ. If he gets it through, it'll be easy to > add on afterward. > > For the .css, you mean merging autocomplete.css into opac.css ? Couldn't > that be a RM thing at the end, if really needed? Otherwise that makes the > rebasing very painful. Ivan is certainly not at ease with that demand. The RM will generate the scss files, you don't need to submit those. But they won't merge CSS changes into the css files. But the risk of conflict is not usually high there.
Created attachment 120028 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also).
Created attachment 120029 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also).
Created attachment 120084 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test, follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results.
Created attachment 120088 [details] GIF showing the buggy behavior I have to mark this "Failed QA" again because the QA tool issues were not all addressed. Please run the QA tool and fix the problems it raises. The only other thing I noticed in testing today was that there is a change in behavior of the search tabs in the staff client. The layout is off, and the focus doesn't move to the input field like it does on the other tabs. See the attached screen grab.
(In reply to Owen Leonard from comment #21) > Created attachment 120088 [details] > GIF showing the buggy behavior > > I have to mark this "Failed QA" again because the QA tool issues were not > all addressed. Please run the QA tool and fix the problems it raises. > > The only other thing I noticed in testing today was that there is a change > in behavior of the search tabs in the staff client. The layout is off, and > the focus doesn't move to the input field like it does on the other tabs. > See the attached screen grab. You have changed/moved CSS files and it provoke your problem with the layout (it was not good idea). Katrin Fischer gave you notice of that! Thanks!
(In reply to Ivan Dziuba from comment #22) > You have changed/moved CSS files and it provoke your problem with the layout Okay, but the issue with the input field focus is the same without my follow-up.
(In reply to Owen Leonard from comment #21) > The layout is off My mistake--I recorded that screengrab without having rebuilt the CSS from SCSS. I will modify the test plan to make that part clear.
Created attachment 120109 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results.
Created attachment 120140 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also).
Created attachment 120141 [details] [review] Bug 27113: (follow-up) QA Tests Bug 27113: (follow-up) QA Tests (2)
> > You have changed/moved CSS files and it provoke your problem with the layout > (it was not good idea). Katrin Fischer gave you notice of that! > Thanks! I think I am the one who messed up here. Owen is our GUI expert, what he says goes :) I think where I got confused was the file endings. It should have been: RM does generate .css files, but you should commit the .scss changes. (Basically, the fiels that are minified and generated - don't submit them, but the normal ones you need to). Owen, please feel free to correct if I got it not quite right again.
Created attachment 120806 [details] [review] Bug 27113: (follow-up) QA Tests Bug 27113: (follow-up) QA Tests (2) Bug 27113: (follow-up) Fixup warnings. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also).
Created attachment 121497 [details] [review] I have changed CGI::Session->new() from CGI::Session->load() for use session. This condition - 'CGI::Session->load() or die CGI::Session->errstr();' I am not sure that was good idea to use 'die' in this condition...What do you think? Thanks! Bug 27113: (follow-up) Correction with Session TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also)
Created attachment 121498 [details] [review] Bug 27113: (follow-up) Correction with Session I have changed CGI::Session->new() from CGI::Session->load() for use session. This condition - 'CGI::Session->load() or die CGI::Session->errstr();' I am not sure that was good idea to use 'die' in this condition...What do you think? Thanks!
Patch no longer applies 8-(... Bug 27113 - ElasticSearch: Autocomplete in input search 120109 - Bug 27113: (follow-up) Move new CSS to main SCSS files 120140 - Bug 27113: ElasticSearch: Autocomplete in input search 120806 - Bug 27113: (follow-up) QA Tests 121498 - Bug 27113: (follow-up) Correction with Session Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 27113: (follow-up) Move new CSS to main SCSS files error: sha1 information is lacking or useless (koha-tmpl/intranet-tmpl/prog/css/elasticsearch/autocomplete.css). error: could not build fake ancestor Patch failed at 0001 Bug 27113: (follow-up) Move new CSS to main SCSS files
(In reply to David Nind from comment #32) > Patch no longer applies 8-(... > > Bug 27113 - ElasticSearch: Autocomplete in input search > > 120109 - Bug 27113: (follow-up) Move new CSS to main SCSS files > 120140 - Bug 27113: ElasticSearch: Autocomplete in input search > 120806 - Bug 27113: (follow-up) QA Tests > 121498 - Bug 27113: (follow-up) Correction with Session > > Apply? [(y)es, (n)o, (i)nteractive] y > Applying: Bug 27113: (follow-up) Move new CSS to main SCSS files > error: sha1 information is lacking or useless > (koha-tmpl/intranet-tmpl/prog/css/elasticsearch/autocomplete.css). > error: could not build fake ancestor > Patch failed at 0001 Bug 27113: (follow-up) Move new CSS to main SCSS files I checked it and it works :). Do you use Koha 21.05? Thanks. Bug 27113 - ElasticSearch: Autocomplete in input search 120109 - Bug 27113: (follow-up) Move new CSS to main SCSS files 120140 - Bug 27113: ElasticSearch: Autocomplete in input search 120806 - Bug 27113: (follow-up) QA Tests 121498 - Bug 27113: (follow-up) Correction with Session Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 27113: (follow-up) Move new CSS to main SCSS files Using index info to reconstruct a base tree... A koha-tmpl/intranet-tmpl/prog/css/elasticsearch/autocomplete.css M koha-tmpl/intranet-tmpl/prog/en/includes/doc-head-close.inc A koha-tmpl/opac-tmpl/bootstrap/css/opac-elasticsearch/opac-autocomplete.css M koha-tmpl/opac-tmpl/bootstrap/css/src/opac.scss M koha-tmpl/opac-tmpl/bootstrap/en/includes/doc-head-close.inc Falling back to patching base and 3-way merge... Auto-merging koha-tmpl/opac-tmpl/bootstrap/css/src/opac.scss Applying: Bug 27113: ElasticSearch: Autocomplete in input search Applying: Bug 27113: (follow-up) QA Tests Applying: Bug 27113: (follow-up) Correction with Session
I agree with Fridolin: we should think of adding API routes for this. There should be a privileged route, and an unprivileged route. You shouldn't worry about sessions and all. Just add the required permissions to interact with the routes.
I'm sorry Tomas, this really muddied the picture for Ivan, and I can't help him as I've no idea how he should proceed. Do you mean his api/elasticsearch/elasticsearch.pl should be put under /v1/ and the routes added to swagger, etc... Any wiki to help him there? We had hope this would get into 21.05 initially, but right now it looks like Ivan will recode it for the foreseable future. Which won't happen, I'll give him one more go at it then drop it.
(In reply to Tomás Cohen Arazi from comment #34) > I agree with Fridolin: we should think of adding API routes for this. There > should be a privileged route, and an unprivileged route. > You shouldn't worry about sessions and all. Just add the required > permissions to interact with the routes. Yeah, I don't get why session data is being used here. I don't even know if there needs to be 2 routes. Maybe just 1 public/unprivileged API route.
Also, it's not clear from the test plan what the user is expected to see/do in terms of the autocomplete...
(In reply to Ivan Dziuba from comment #33) .. > I checked it and it works :). Do you use Koha 21.05? Thanks. .. Hi Ivan. I'm using master - I tried again and still get the same error for the CSS follow-up patch: .. Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 27113: (follow-up) Move new CSS to main SCSS files error: sha1 information is lacking or useless (koha-tmpl/intranet-tmpl/prog/css/elasticsearch/autocomplete.css). error: could not build fake ancestor Patch failed at 0001 Bug 27113: (follow-up) Move new CSS to main SCSS files .. I can get the patches to apply if I use interactive, comment out the CSS follow-up patch, then apply again using interactive and then just apply the CSS follow-up patch. I'm assuming the error is because the follow-up CSS is being applied first, but what it is changing doesn't exist yet as it is one of the later patches. I don't know enough about patches to fix this up, but will try and test with what I have.
Created attachment 121555 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 121556 [details] [review] Bug 27113: (follow-up) QA Tests Bug 27113: (follow-up) QA Tests (2) Bug 27113: (follow-up) Fixup warnings. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 121557 [details] [review] Bug 27113: (follow-up) Correction with Session I have changed CGI::Session->new() from CGI::Session->load() for use session. This condition - 'CGI::Session->load() or die CGI::Session->errstr();' I am not sure that was good idea to use 'die' in this condition...What do you think? Thanks! Signed-off-by: David Nind <david@davidnind.com>
Created attachment 121558 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com>
I finally got this to work for me, and it is a really nice feature. Nice work! I've signed off, but these are my suggestions for the wording for the new system preferences - the QA Team will probably comment on this is as well, so you may as well get ahead of them!: 1. IntranetAutocompleteElasticSearch: change system preference name to StaffInterfaceAutocompleteElasticsearch - We are using 'staff interface' now, instead of 'intranet' - not sure what the protocol is for system preference names, but for new system preferences we should probably start using StaffInterface; Elasticsearch doesn't have a capital S (see for example the SearchEngine system preference) 2. Wording for IntranetAutocompleteElasticSearch: . Currently: Don't show/Show looking terms based on a provided text by using an ElasticSearch for Intranet. . Suggested: Don't show/Show search suggestions in the staff interface when using Elasticsearch. 3. Wording for OPACAutocompleteElasticSearch: . Currently: Don't show/Show looking terms based on a provided text by using an ElasticSearch for OPAC. . Change to: Don't show/Show search suggestions in the OPAC when using Elasticsearch. Other testing notes (using koha-testing-docker): . Package command to rebuild the index: koha-elasticsearch -d --rebuild -a -b kohadev . Needed to change .env file as a result of recent commits to koha-testing-docker for Elasticsearch (compared default with custom .env file and added new variables) . As indicated earlier, had to apply patch interactively: 1st excluding CSS patch, then again to include CSS patch . Rebuild CSS as per CSS patch
I am tempted to mark this as Failed QA, since I'm pretty sure the session handling doesn't work as intended (I think it's saving the session data in a file rather than in the database), but I haven't actually tested it, so leaving it as is.
As QA: I really like David's suggestions for the rephrasing. I think it makes the use much more clear. But before spending time on these, we probably need to sort out the other requirements/comments. Sessions: * David, can you be a bit more precise about what you think doesn't work? * Tomas, can you help with what you imagined and maybe the sessions bit? Does this need to be fixed here or could it be a follow-up? (In reply to Tomás Cohen Arazi from comment #34) > I agree with Fridolin: we should think of adding API routes for this. There > should be a privileged route, and an unprivileged route. > You shouldn't worry about sessions and all. Just add the required > permissions to interact with the routes.
Ivan has already recoded it without sessions in our codebase. But before pushing that change, we'll be waiting for Tomas' instructions regarding the svc/
(In reply to Blou from comment #46) > Ivan has already recoded it without sessions in our codebase. But before > pushing that change, we'll be waiting for Tomas' instructions regarding the > svc/ Hi Blou, thx for the update! So we can scratch that off the list. I'll ask Tomas to have another look.
(In reply to Blou from comment #46) > Ivan has already recoded it without sessions in our codebase. But before > pushing that change, we'll be waiting for Tomas' instructions regarding the > svc/ I'll mostly leave it to Tomas, but it might be worth looking over https://wiki.koha-community.org/wiki/Rest_Api_HowTo Basically, the "svc/" API scripts are legacy, so we don't want to be adding new ones. We just want to build up the Mojolicious-based REST API.
(In reply to Blou from comment #46) > Ivan has already recoded it without sessions in our codebase. But before > pushing that change, we'll be waiting for Tomas' instructions regarding the > svc/ I think this is a subject worth discussing in the open, i.e. in koha-devel. As I mentioned (not sure where) I believe we could try adding some filtering capabilities to GET /biblios to accomplish this. I proposed something similar on bug 25870. In that case, I proposed a 'q_ccl' query param that could take a CCL query so we could accept complex queries. The route would then extract the ids (biblionumbers) and return Koha::Biblios->to_api objects as required. I don't think CCL is a cool feature, but it felt like good start: allowing people to query records through the API. The ES syntax might be a high constraint for the feature (leaving Zebra users out), specially if you are querying things like title, author, etc. But that's another possibility: add a query parameter that only works with ES (q_es?). Regarding routes, you need to think what the response would look like. And it will help you figure what you want. Is it ok/enough for GET /biblios?<filter> to return JSON representation of the biblio+biblioitems table columns? Unfortunately, the OpenAPI v2.0 schema doesn't provide a way to define different response schemas for different content-type responses, and thus we left GET /biblios/:biblio_id under-documented (it is supported on v3) but you can try GET /biblios/1 and see the response to get an idea of what we already return [1]. It feels like we need to build a good query language, and reuse this routes. [1] This is how we build the JSON representation of a biblio https://git.koha-community.org/Koha-community/Koha/src/branch/master/Koha/Biblio.pm#L867-L892
(In reply to Tomás Cohen Arazi from comment #49) > (In reply to Blou from comment #46) > > Ivan has already recoded it without sessions in our codebase. But before > > pushing that change, we'll be waiting for Tomas' instructions regarding the > > svc/ > > I think this is a subject worth discussing in the open, i.e. in koha-devel. > > As I mentioned (not sure where) I believe we could try adding some filtering > capabilities to GET /biblios to accomplish this. I proposed something > similar on bug 25870. In that case, I proposed a 'q_ccl' query param that > could take a CCL query so we could accept complex queries. The route would > then extract the ids (biblionumbers) and return Koha::Biblios->to_api > objects as required. > > I don't think CCL is a cool feature, but it felt like good start: allowing > people to query records through the API. > > The ES syntax might be a high constraint for the feature (leaving Zebra > users out), specially if you are querying things like title, author, etc. > But that's another possibility: add a query parameter that only works with > ES (q_es?). > > Regarding routes, you need to think what the response would look like. And > it will help you figure what you want. Is it ok/enough for GET > /biblios?<filter> to return JSON representation of the biblio+biblioitems > table columns? Unfortunately, the OpenAPI v2.0 schema doesn't provide a way > to define different response schemas for different content-type responses, > and thus we left GET /biblios/:biblio_id under-documented (it is supported > on v3) but you can try > > GET /biblios/1 > > and see the response to get an idea of what we already return [1]. > > It feels like we need to build a good query language, and reuse this routes. > > [1] This is how we build the JSON representation of a biblio > https://git.koha-community.org/Koha-community/Koha/src/branch/master/Koha/ > Biblio.pm#L867-L892 I don't understand what is it means `It feels like we need to build a good query language, and reuse these routes.`? Here I use dynamic queries $q, $prefix and a kind of "$token" to understand for which query hints were returned and whether to show or skip them. (depends on the speed of writing). Could you please clarify the structure of URL for that. Thanks!
Created attachment 131207 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 131208 [details] [review] Bug 27113: (follow-up) QA Tests Bug 27113: (follow-up) QA Tests (2) Bug 27113: (follow-up) Fixup warnings. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 131209 [details] [review] Bug 27113: (follow-up) Correction with Session I have changed CGI::Session->new() from CGI::Session->load() for use session. This condition - 'CGI::Session->load() or die CGI::Session->errstr();' I am not sure that was good idea to use 'die' in this condition...What do you think? Thanks! Signed-off-by: David Nind <david@davidnind.com>
Created attachment 131210 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com>
Created attachment 131211 [details] [review] Bug 27113: Rebase and correct autocomplete appearance Rebase of previous patches and correct the buggy behavior (See "GIF showing buggy behavior attachment") Test plan: 1- Search for an item. Notice that the autocomplete functionnality is not available. (See "Result" attachment) 2- Make sure that you are using Elasticsearch as your searching engine. (Preference->SearchEngine->ElasticSearch) 3- Apply the patch. 4- Run ./installer/data/mysql/updatedatabase.pl 5- Looks for the following preferences : - IntranetAutocompleteElasticSearch - OPACAutocompleteElasticSearch 6- Set their value to show. 7- Rebuild the OPAC and staff client CSS (This step is important because the autocomplete CSS is moved to the main SCSS file): Instructions here: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client 8- Search for an item and notice that the autocomplete functionnality is now available. 9- In the staff interface, go in "Search the catalog". 10- Notice that the buggy behavior is now fixed.
Created attachment 132360 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 132361 [details] [review] Bug 27113: (follow-up) QA Tests Bug 27113: (follow-up) QA Tests (2) Bug 27113: (follow-up) Fixup warnings. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 132362 [details] [review] Bug 27113: (follow-up) Correction with Session I have changed CGI::Session->new() from CGI::Session->load() for use session. This condition - 'CGI::Session->load() or die CGI::Session->errstr();' I am not sure that was good idea to use 'die' in this condition...What do you think? Thanks! Signed-off-by: David Nind <david@davidnind.com>
Created attachment 132363 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com>
Created attachment 132364 [details] [review] Bug 27113: Rebase and correct autocomplete appearance Rebase of previous patches and correct the buggy behavior (See "GIF showing buggy behavior attachment") Test plan: 1- Search for an item. Notice that the autocomplete functionnality is not available. (See "Result" attachment) 2- Make sure that you are using Elasticsearch as your searching engine. (Preference->SearchEngine->ElasticSearch) 3- Apply the patch. 4- Run ./installer/data/mysql/updatedatabase.pl 5- Looks for the following preferences : - IntranetAutocompleteElasticSearch - OPACAutocompleteElasticSearch 6- Set their value to show. 7- Rebuild the OPAC and staff client CSS (This step is important because the autocomplete CSS is moved to the main SCSS file): Instructions here: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client 8- Search for an item and notice that the autocomplete functionnality is now available. 9- In the staff interface, go in "Search the catalog". 10- Notice that the buggy behavior is now fixed.
Created attachment 132808 [details] [review] Bug 27113: No autosubmit after clicking autocomplete option in advanced search This patch makes it so it doesn't automatically submit form after clicking an autocomplete option in advanced search. Also removes autocomplete options after pressing tab.
Created attachment 132945 [details] [review] Bug 27113: Removes items in OPACHiddenItems from autocomplete Items specified in OPACHiddenItems preference are removed from search suggestions in OPAC. Test plan: 1- Search for an item. Notice that the autocomplete functionnality is not available. (See "Result" attachment) 2- Make sure that you are using Elasticsearch as your searching engine. (Preference->SearchEngine->ElasticSearch) 3- Apply the patch. 4- Run ./installer/data/mysql/updatedatabase.pl 5- Looks for the following preferences : - IntranetAutocompleteElasticSearch - OPACAutocompleteElasticSearch 6- Set their value to show. 7- Rebuild the OPAC and staff client CSS (This step is important because the autocomplete CSS is moved to the main SCSS file): Instructions here: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client 8- Search for an item and notice that the autocomplete functionnality is now available. 9- In the staff interface, go in "Search the catalog". 10- Notice that the buggy behavior is now fixed. 11- Search for an item with basic search and click on one of the options. 12- Notice it automatically sends you to the selected items page. 13- This time, search an item using the advanced search and click on one of the options. 14- Notice it just writes the chosen option in the input instead. 15- Go to Administration -> system preferences -> OPACHiddenItems. 16- Define a rule that applies to an existing item. 17- Search that item in OPAC and notice it doesn't give you the suggestion for that item.
My concern is that searching with more than 1 prefix is going to be too slow when there are lots of items since it will go through a triple nested loop.
Sorry. I meant double nested loops.
Created attachment 132946 [details] [review] Bug 27113: Removes items in OPACHiddenItems from autocomplete Items specified in OPACHiddenItems preference are removed from search suggestions in OPAC. Test plan: 1- Search for an item. Notice that the autocomplete functionnality is not available. (See "Result" attachment) 2- Make sure that you are using Elasticsearch as your searching engine. (Preference->SearchEngine->ElasticSearch) 3- Apply the patch. 4- Run ./installer/data/mysql/updatedatabase.pl 5- Looks for the following preferences : - IntranetAutocompleteElasticSearch - OPACAutocompleteElasticSearch 6- Set their value to show. 7- Rebuild the OPAC and staff client CSS (This step is important because the autocomplete CSS is moved to the main SCSS file): Instructions here: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client 8- Search for an item and notice that the autocomplete functionnality is now available. 9- In the staff interface, go in "Search the catalog". 10- Notice that the buggy behavior is now fixed. 11- Search for an item with basic search and click on one of the options. 12- Notice it automatically sends you to the selected items page. 13- This time, search an item using the advanced search and click on one of the options. 14- Notice it just writes the chosen option in the input instead. 15- Go to Administration -> system preferences -> OPACHiddenItems. 16- Define a rule that applies to an existing item. 17- Search that item in OPAC and notice it doesn't give you the suggestion for that item.
Created attachment 133042 [details] [review] Bug 27113: Removes hidden and suppressed items from OPAC autocomplete Items specified in OPACHiddenItems preference and items affected by OpacSuppression are removed from search suggestions in OPAC. Test plan: 1- Search for an item. Notice that the autocomplete functionnality is not available. (See "Result" attachment) 2- Make sure that you are using Elasticsearch as your searching engine. (Preference->SearchEngine->ElasticSearch) 3- Apply the patch. 4- Run ./installer/data/mysql/updatedatabase.pl 5- Looks for the following preferences : - IntranetAutocompleteElasticSearch - OPACAutocompleteElasticSearch 6- Set their value to show. 7- Rebuild the OPAC and staff client CSS (This step is important because the autocomplete CSS is moved to the main SCSS file): Instructions here: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client 8- Search for an item and notice that the autocomplete functionnality is now available. 9- In the staff interface, go in "Search the catalog". 10- Notice that the buggy behavior is now fixed. 11- Search for an item with basic search and click on one of the options. 12- Notice it automatically sends you to the selected items page. 13- This time, search an item using the advanced search and click on one of the options. 14- Notice it just writes the chosen option in the input instead. 15- Go to Administration -> system preferences -> OPACHiddenItems. 16- Define a rule that applies to an existing item. 17- Search that item in OPAC and notice it doesn't give you the suggestion for that item. 18- Go to Administration -> system preferences -> OpacSuppression and set it to "hide". 19- Add or edit a record and set 942$n field to yes. 20- Search that entry in OPAC and notice it doesn't give you the suggestion for that entry.
Created attachment 133045 [details] [review] Bug 27113: Focus the input after selecting "Search the catalog" Small fix to focus the input after clicking "Search the catalog" in staff interface.
Created attachment 133407 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 133409 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 133410 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com>
I know it's bad form, but since we've been told the functionality requires a re-engineering to use the REST api, a lot of rebase are going to happen until we get this on the master, so I squashed it all. Well, not all, I kept the one patch contributed from outside our company.
Created attachment 136312 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com> Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com>
Squashed the two patches at the request of Blou.
Nevermind. Put back the last two patches.
Created attachment 136456 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 136457 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com>
Created attachment 136464 [details] [review] Bug 27113: (follow-up) Update descriptions for system preferences Update the descriptions for the new system preferences - IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch. Test plan: 1. Review the descriptions for the IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch system preferences. 2. Make sure they are readable and make sense. Signed-off-by: David Nind <david@davidnind.com>
Created attachment 136465 [details] Bug 27113 - Advanced search - more options display issue I've added a follow-up with my suggestions for the descriptions for the two new system preferences. I've changed the status to Failed QA as there is a slight issue with the staff interface advanced search > more options after the patch is applied. See the attached screenshots. Also, something for QA to comment on. The spelling for Elasticsearch - the s in search is not capitalised, as it is the product name (not ElasticSearch). Does this need changing for the preference names, and the other code in the patches? Can confirm everything continues to work otherwise, and I like it!
Created attachment 136653 [details] [review] Bug 27113: (follow-up) fix adv. searches - more options display issue
Fixed the little display issue. Also, does anyone know how to have the same relative path to one file from both OPAC and staff interface url? (e.g. http://master/api/elasticsearch/elasticsearch.pl and http://master-intranet/api/elasticsearch/elasticsearch.pl) With this we would only need to create one .pl file to be used by both interface instead of one for each.
Created attachment 136657 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 136658 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 136659 [details] [review] Bug 27113: (follow-up) Update descriptions for system preferences Update the descriptions for the new system preferences - IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch. Test plan: 1. Review the descriptions for the IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch system preferences. 2. Make sure they are readable and make sense. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 136660 [details] [review] Bug 27113: (follow-up) fix adv. searches - more options display issue Signed-off-by: David Nind <david@davidnind.com>
(In reply to Shi Yao Wang from comment #80) > Fixed the little display issue. Thanks! > Also, does anyone know how to have the same relative path to one file from > both OPAC and staff interface url? (e.g. > http://master/api/elasticsearch/elasticsearch.pl and > http://master-intranet/api/elasticsearch/elasticsearch.pl) > > With this we would only need to create one .pl file to be used by both > interface instead of one for each. Sorry, I don't.
> > Also, does anyone know how to have the same relative path to one file from > > both OPAC and staff interface url? (e.g. > > http://master/api/elasticsearch/elasticsearch.pl and > > http://master-intranet/api/elasticsearch/elasticsearch.pl) > > > > With this we would only need to create one .pl file to be used by both > > interface instead of one for each. > > Sorry, I don't. AFAIK it's not possible to share files between OPAC and staff interface.
You can with REST API. Public routes are same file between OPAC and staff : https://wiki.koha-community.org/wiki/REST_api_RFCs#Unprivileged_.28public.29_routes
Created attachment 137436 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 137437 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 137438 [details] [review] Bug 27113: (follow-up) Update descriptions for system preferences Update the descriptions for the new system preferences - IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch. Test plan: 1. Review the descriptions for the IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch system preferences. 2. Make sure they are readable and make sense. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 137439 [details] [review] Bug 27113: (follow-up) fix adv. searches - more options display issue Signed-off-by: David Nind <david@davidnind.com>
Fixed an encoding/display problem when using "é", "ù", etc. (In reply to Fridolin Somers from comment #87) > You can with REST API. > Public routes are same file between OPAC and staff : > https://wiki.koha-community.org/wiki/REST_api_RFCs#Unprivileged_.28public. > 29_routes I don't know how to do it, but I think it would be a good idea.
First of all: I quite like the functionality and how this looks. It also appears to be pretty fast and accurate as far as I can tell. I especially like that it's not only available for the keyword index, but also on other search options like author search and limits the results down accordingly. That said: There are a couple things here that would require fixing and a big question we need to resolve for moving this forward. The small stuff: 1) In the database update, there should still be a description for the system preference. 2) sysprefs.sql is sorted alphabetically to avoid conflicts, please move the prefs into the correct places. 3) Instead of use strict; use warnings; we use use Modern::Perl; (see PERL2 in coding guidelines) 4) In the database update elasticsearch should be Elasticsearch. Somewhat bigger: 5) I have problems making this work with the "library catalog" and "title" search options in simple search in OPAC. title doesn't give me any results and library catalog looks like author (but it's a keyword search). Something might not be 'wired' correctly there. 6) Updating installations: I believe you also need to reset mappings to make this work? How can we ensure this works out of the box for people updating without them having to "redo" their index configuration? (database update?) 7) sub filterAutocomplete looks complex and I wonder if it should be in the .pl and not in a module where it can be tested. 942 is hardcoded as field for OpacSuppression, which is a little risky. I wonder if it could be more elegant to filter this at search time instead of after? The last big question, I'll refer to the RM: This adds to /svc which is the outdated way of doing things as others have pointed out before. I think in order to move on here and before more time is spend, we should come to a decision of what is required for this patch to make it in and if it needs to be rewritten to use REST API or if this could be an exception.
Created attachment 137780 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 137781 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 137782 [details] [review] Bug 27113: (follow-up) Update descriptions for system preferences Update the descriptions for the new system preferences - IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch. Test plan: 1. Review the descriptions for the IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch system preferences. 2. Make sure they are readable and make sense. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 137783 [details] [review] Bug 27113: (follow-up) fix adv. searches - more options display issue Signed-off-by: David Nind <david@davidnind.com>
(In reply to Katrin Fischer from comment #93) > First of all: I quite like the functionality and how this looks. It also > appears to be pretty fast and accurate as far as I can tell. I especially > like that it's not only available for the keyword index, but also on other > search options like author search and limits the results down accordingly. > > That said: There are a couple things here that would require fixing and a > big question we need to resolve for moving this forward. > > > The small stuff: > 1) In the database update, there should still be a description for the > system preference. > 2) sysprefs.sql is sorted alphabetically to avoid conflicts, please move the > prefs into the correct places. > 3) Instead of use strict; use warnings; we use use Modern::Perl; (see PERL2 > in coding guidelines) > 4) In the database update elasticsearch should be Elasticsearch. > > > Somewhat bigger: > > 5) I have problems making this work with the "library catalog" and "title" > search options in simple search in OPAC. title doesn't give me any results > and library catalog looks like author (but it's a keyword search). Something > might not be 'wired' correctly there. > 6) Updating installations: I believe you also need to reset mappings to make > this work? How can we ensure this works out of the box for people updating > without them having to "redo" their index configuration? (database update?) > 7) sub filterAutocomplete looks complex and I wonder if it should be in the > .pl and not in a module where it can be tested. 942 is hardcoded as field > for OpacSuppression, which is a little risky. I wonder if it could be more > elegant to filter this at search time instead of after? Small stuff fixed. 5) Searching with "title" works for me. Maybe an index issue? or is it just me? "library catalog" search displays suggestions for each search options in this order: title-cover, author, subject, title-series, publisher. What I think probably happened is, since "title" search doesn't work for you, it showed "author" suggestions where you had more match than the maximum number of rows displayed. 6) I don't know :/ 7) I can move it to a module, but which one? Maybe in a new module? if so, where? I reused the code in opac/opac-detail.pl where 942 is hardcoded too. I do think it would be better to filter at search time but I don't know how...
Created attachment 139624 [details] [review] Bug 27113: (follow-up) replace removed GetMarcBiblio
Created attachment 141466 [details] [review] Bug 27113: Allow for ending special characters in titles Modifies the cleanup regex that removed )}' and " endings from the suggested titles or authors. When selected without those, the record was not found. Ex: "MyTitle (1234)" became "MyTitle (123" in the list. It returned no results, where the full title or "MyTitle 123" would.
Comments from Kate/RM would be appreciated so that Shi Yao (or others) can move ahead with this patch. ShiYao has fixed and commented on the bullet points and this is now in Limbo.
Created attachment 145167 [details] [review] Bug 27113: (follow-up) diacritics/acccents in sought word would make autocomplete return nothing
Created attachment 146015 [details] [review] Bug 27113: Move filterAutocomplete to a new module and filter the result at search time
sub filterAutocomplete moved to a new module and now the filtering is done at the search time. (In reply to Katrin Fischer from comment #93) > 6) Updating installations: I believe you also need to reset mappings to make > this work? How can we ensure this works out of the box for people updating > without them having to "redo" their index configuration? (database update?) It's been wrote in the test plan by precaution but there is no need reset mappings to make this work.
Waiting for some work on bug 25870 and bug 30790 to continue this.
Created attachment 159769 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 159770 [details] [review] Bug 27113: (follow-up) Update descriptions for system preferences Update the descriptions for the new system preferences - IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch. Test plan: 1. Review the descriptions for the IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch system preferences. 2. Make sure they are readable and make sense. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 159771 [details] [review] Bug 27113: (follow-up) fix adv. searches - more options display issue Signed-off-by: David Nind <david@davidnind.com>
Created attachment 159772 [details] [review] Bug 27113: (follow-up) replace removed GetMarcBiblio
Created attachment 159773 [details] [review] Bug 27113: Allow for ending special characters in titles Modifies the cleanup regex that removed )}' and " endings from the suggested titles or authors. When selected without those, the record was not found. Ex: "MyTitle (1234)" became "MyTitle (123" in the list. It returned no results, where the full title or "MyTitle 123" would.
Created attachment 159774 [details] [review] Bug 27113: (follow-up) diacritics/acccents in sought word would make autocomplete return nothing
Created attachment 159775 [details] [review] Bug 27113: Move filterAutocomplete to a new module and filter the result at search time
Rebased
Created attachment 160134 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 160135 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 160136 [details] [review] Bug 27113: (follow-up) Update descriptions for system preferences Update the descriptions for the new system preferences - IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch. Test plan: 1. Review the descriptions for the IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch system preferences. 2. Make sure they are readable and make sense. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 160137 [details] [review] Bug 27113: (follow-up) fix adv. searches - more options display issue Signed-off-by: David Nind <david@davidnind.com>
Created attachment 160138 [details] [review] Bug 27113: (follow-up) replace removed GetMarcBiblio
Created attachment 160139 [details] [review] Bug 27113: Allow for ending special characters in titles Modifies the cleanup regex that removed )}' and " endings from the suggested titles or authors. When selected without those, the record was not found. Ex: "MyTitle (1234)" became "MyTitle (123" in the list. It returned no results, where the full title or "MyTitle 123" would.
Created attachment 160140 [details] [review] Bug 27113: (follow-up) diacritics/acccents in sought word would make autocomplete return nothing
Created attachment 160141 [details] [review] Bug 27113: Move filterAutocomplete to a new module and filter the result at search time
Created attachment 160142 [details] Bug 27113: Fix the Input Size of the Catalog Search Bar This patch addresses the sizing issue with the catalog search bar, ensuring that the input zone now covers the entire white bar area. TEST PLAN: 1) Apply the patches that precede this one. 2) Activate IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch in the system preferences. 2) Click "Catalog search" in the navigation bar. 3) Click on the right side of the white area of the search bar. 4) Confirm that it does not allow you to start typing unless you click on the left side of the search bar. 5) Apply this patch. 6) Repeat steps 2 and 3. 7) Verify that this time it allows you to type in the search bar even if you click on the right side of the searc h bar.
(In reply to Katrin Fischer from comment #93) > The last big question, I'll refer to the RM: > This adds to /svc which is the outdated way of doing things as others have > pointed out before. I think in order to move on here and before more time is > spend, we should come to a decision of what is required for this patch to > make it in and if it needs to be rewritten to use REST API or if this could > be an exception. As far as I can see, the patches are still adding to svc, not the REST API? So I guess we need to resolve this question for the bug to move forward?
I believe we had a plan here involving solving bug 25870 first as a better base for the work here. It's already linked as a dependency, but needs some more work right now. I know a lot of people would love to see this functionality and I hope we can figure out the implementation bits this cycle.
Created attachment 161076 [details] Bug 27113: Fix the Input Size of the Catalog Search Bar
Created attachment 161077 [details] [review] Bug 27113: Fix the Input Size of the Catalog Search Bar This patch addresses the sizing issue with the catalog search bar, ensuring that the input zone now covers the entire white bar area. TEST PLAN: 1) Apply the patches that precede this one. 2) Activate IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch in the system preferences. 2) Click "Catalog search" in the navigation bar. 3) Click on the right side of the white area of the search bar. 4) Confirm that it does not allow you to start typing unless you click on the left side of the search bar. 5) Apply this patch. 6) Repeat steps 2 and 3. 7) Verify that this time it allows you to type in the search bar even if you click on the right side of the searc h bar.
(In reply to Katrin Fischer from comment #124) > I believe we had a plan here involving solving bug 25870 first as a better > base for the work here. It's already linked as a dependency, but needs some > more work right now. > > I know a lot of people would love to see this functionality and I hope we > can figure out the implementation bits this cycle. Can you have a look at bug 25870 maybe?
Created attachment 163826 [details] [review] Bug 27113: Fix the issue of the cursor not displaying
Blocked by bug 25870
Created attachment 165895 [details] [review] Bug 27113: ElasticSearch: Autocomplete in input search Usually the user knows only part of the title of the book or only the name of the author, etc. When he start search something Koha (ElasticSearch) predicts the rest of a word or expression which user is typing. Autocomplete predicts that thanks to index of ElasticSearch. TEST PLAN Important! In this patch we need to do reindex ElasticSearch. ElasticSearch must have all information in his index. 1. Go Intranet -> Preference -> SearchEngine -> ElasticSearch !! APPLY PATCH !! 2. Mapping is good (Intranet -> Catalog -> Search engine configuration (Elasticsearch) ). !Recommended 'Reset Mapping' -> 'Yes' 3. In your koha-conf.xml file you must have good <index_name> for <elasticsearch> and version ES 4. Update Preference: ./installer/data/mysql/updatedatabase.pl If that passe good you can look the lines: - DEV atomic update: bug_27113-elasticsearch_autocomplete_input_search.perl - Upgrade to XXX done : Bug 27113 - Autocomplete input on main page with elasticsearch 5. After that we can look two options in the preferences: - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; 4. For add information in the index we must run script for reindexing: ./misc/search_tools/rebuild_elasticsearch.pl -v -d 5. Waiting for the end of indexing 6. Go on Preference and find : - IntranetAutocompleteElasticSearch; - OPACAutocompleteElasticSearch; Value "Show" turn on autocomplete. 7. Now we have Autocomplete for Intranet/OPAC search input (advanced search also). Signed-off-by: David Nind <david@davidnind.com>
Created attachment 165896 [details] [review] Bug 27113: (follow-up) Move new CSS to main SCSS files This patch removes the separate CSS files added for the autocomplete feature and puts them into the "main" SCSS files. I think a separate file isn't necessary because the amount of CSS it adds is so small. I've also tweaked the style of the autocomplete menu shown when you use the arrow keys to navigate through the autocomplete choices. I think the previous white-on-light-green didn't have enough contrast. To test you must rebuild the OPAC and staff client CSS: https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client Follow the previous test plan, being careful to observe how it works when the autocomplete menu has been triggered and you use the arrow keys to navigate through the results. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 165897 [details] [review] Bug 27113: (follow-up) Update descriptions for system preferences Update the descriptions for the new system preferences - IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch. Test plan: 1. Review the descriptions for the IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch system preferences. 2. Make sure they are readable and make sense. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com>
Created attachment 165898 [details] [review] Bug 27113: (follow-up) fix adv. searches - more options display issue Signed-off-by: David Nind <david@davidnind.com>
Created attachment 165899 [details] [review] Bug 27113: (follow-up) replace removed GetMarcBiblio
Created attachment 165900 [details] [review] Bug 27113: Allow for ending special characters in titles Modifies the cleanup regex that removed )}' and " endings from the suggested titles or authors. When selected without those, the record was not found. Ex: "MyTitle (1234)" became "MyTitle (123" in the list. It returned no results, where the full title or "MyTitle 123" would.
Created attachment 165901 [details] [review] Bug 27113: (follow-up) diacritics/acccents in sought word would make autocomplete return nothing
Created attachment 165902 [details] [review] Bug 27113: Move filterAutocomplete to a new module and filter the result at search time
Created attachment 165903 [details] [review] Bug 27113: Fix the Input Size of the Catalog Search Bar This patch addresses the sizing issue with the catalog search bar, ensuring that the input zone now covers the entire white bar area. TEST PLAN: 1) Apply the patches that precede this one. 2) Activate IntranetAutocompleteElasticSearch and OPACAutocompleteElasticSearch in the system preferences. 2) Click "Catalog search" in the navigation bar. 3) Click on the right side of the white area of the search bar. 4) Confirm that it does not allow you to start typing unless you click on the left side of the search bar. 5) Apply this patch. 6) Repeat steps 2 and 3. 7) Verify that this time it allows you to type in the search bar even if you click on the right side of the searc h bar.
Created attachment 165904 [details] [review] Bug 27113: Fix the issue of the cursor not displaying
rebase
Created attachment 165913 [details] [review] Bug 27113: Fix input size of all searchbar
Created attachment 165922 [details] [review] Bug 27113: Fix input size of all searchbar