Description
Nick Clemens (kidclamp)
2017-07-17 20:20:03 UTC
Created attachment 65086 [details] [review] Bug 18948 - Elasticsearch - Reindexes should use aliases to avoid down time while reindexing To test: 1 - Delete your current indices for ES (if any) 2 - Reindex your records 3 - curl -XGET 'localhost:9200/koha_kohadev?pretty' 4 - Note indices are now of form koha_kohadev_########## 5 - The numbers there are a result of appending 'time' command to name 6 - Ensure searching works as before the patches 7 - Reindex your records with a commit setting of 1 (to slow things down) you may need a large numebr fo records 8 - Ensure searching works during reindex 9 - Ensure reindexing completes successfully 10 - curl -XGET 'localhost:9200/koha_kohadev?pretty' 11 - There should only be one index each for biblios and authorities ** - Note number in above 12 - Perform a partial reindex by passing a biblionumber to the command 13 - Curl the indices again and ensure number/name has not changed TODO: 1 - Ensure if records are indexed incrementally before first rebuild that an alias is used 2 - Unit tests Other comments welcome, this was first round proof of concept Created attachment 65147 [details] [review] Bug 18948 - Elasticsearch - Reindexes should use aliases to avoid down time while reindexing Refactoring of rebuild_elastic_search.pl to move code into Koha::SearchEngine::Elasticsearch::Indexer Created attachment 65159 [details] [review] Bug 18948 - Elasticsearch - Reindexes should use aliases to avoid down time while reindexing To test: 1 - Delete your current indices for ES (if any) 2 - Reindex your records 3 - curl 'localhost:9200/_cat/indices?v' 4 - Note indices are now of form koha_kohadev_########## 5 - The numbers there are a result of appending 'time' command to name 6 - Ensure searching works as before the patches 7 - Reindex your records with a commit setting of 1 (to slow things down) you may need a large numebr fo records 8 - Ensure searching works during reindex 9 - Ensure reindexing completes successfully 10 - curl 'localhost:9200/_cat/indices?v' 11 - There should only be one index each for biblios and authorities ** - Note number in above 12 - Perform a partial reindex by passing a biblionumber to the command 13 - Curl the indices again and ensure number/name has not changed 14 - Delete all indexes curl -XDELETE 'localhost:9200/koha_kohadev_*?pretty' 15 - Edit a known biblio: http://localhost:8081/cgi-bin/koha/cataloguing/addbiblio.pl?biblionumber=1 16 - Add a word to the title "potato" 17 - Save the record 18 - Search for potato 19 - You should arrive at the detail page for that record 20 - Confirm your index is of form koha_kohadev_########## Needs unit tests Patch does not apply, SHA1 error for Indexer.pm Created attachment 70815 [details] [review] Bug 18948 - Elasticsearch - Reindexes should use aliases to avoid down time while reindexing Refactoring of rebuild_elastic_search.pl to move code into Koha::SearchEngine::Elasticsearch::Indexer Current status: Patch doesn't apply Created attachment 70816 [details] [review] Bug 18948 - Elasticsearch - Reindexes should use aliases to avoid down time while reindexing To test: 1 - Delete your current indices for ES (if any) 2 - Reindex your records 3 - curl 'localhost:9200/_cat/indices?v' 4 - Note indices are now of form koha_kohadev_########## 5 - The numbers there are a result of appending 'time' command to name 6 - Ensure searching works as before the patches 7 - Reindex your records with a commit setting of 1 (to slow things down) you may need a large numebr fo records 8 - Ensure searching works during reindex 9 - Ensure reindexing completes successfully 10 - curl 'localhost:9200/_cat/indices?v' 11 - There should only be one index each for biblios and authorities ** - Note number in above 12 - Perform a partial reindex by passing a biblionumber to the command 13 - Curl the indices again and ensure number/name has not changed 14 - Delete all indexes curl -XDELETE 'localhost:9200/koha_kohadev_*?pretty' 15 - Edit a known biblio: http://localhost:8081/cgi-bin/koha/cataloguing/addbiblio.pl?biblionumber=1 16 - Add a word to the title "potato" 17 - Save the record 18 - Search for potato 19 - You should arrive at the detail page for that record 20 - Confirm your index is of form koha_kohadev_########## Ugly rebase, still needs unit tests and testing to confirm it works I'd like to see an option that uses the existing index also when reindexing all records. In my experience there are cases where you want to see the changes as soon as possible and not wait until the whole indexing process is complete. Testing indexing is just one of them but pretty important during development. Additionally, if aliases are used, the there should be a mechanism to take care of indexing all the changes that were made in staff UI or such during the reindexing process. Otherwise they'd end up existing only in the old index that gets deleted, right? Not working on this currently, leaving it for anyone who wants to look +1 it would be great to use alias for not loosing search results during full reindexing \o/ |