I would be glad to use the rebuild script with some more options like: -n 100 index 100 first records -n "100,2" index 2 records after 100th (101 and 102) --from XXX index biblio with biblionumber >= XXX --to YYY index biblio with biblionumber <= YYY bonus: -i gives elastic install information: syspref, numbers of biblios/auth indexed, name of indice used etc. There is maybe an other way to name the options. Advice or opinion or objection on how to do this ?
It could also be useful to have an option to launch an indexing by decrementing the record number -w "100,2" index 2 biblio with biblionumber after 100th (101 and 102)
It could also be useful to have an option to launch an indexing by decrementing the record number -w "100,2" index 2 biblio with biblionumber after 100th (101 and 102) and -w "100,-2" index 2 biblio with biblionumber before 100th (99 and 98) This allows the highest biblionumbers, i.e. the most recent records, to be indexed first.
I'd say we should use same args as misc/migration_tools/rebuild_zebra.pl : range of records : --length 1234 how many biblio you want to export --offset 1243 offset you want to start to a where limit : --where let you specify a WHERE query, like itemtype='BOOK' or something like that --table specify a table (can be items, biblioitems, biblio, biblio_metadata) to retrieve biblionumber to index.
something whould be awesome : limit on biblionumbers from a file. allow to rebuild from a report results.
Created attachment 89425 [details] [review] Bug 20384 - Elastic rebuild script improvements - options --offset & --length This patch add the options --offset & --length to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help
Created attachment 89559 [details] [review] Bug 20384 - Elastic rebuild script improvements - options --offset & --length This patch add the options --offset & --length to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help
Created attachment 89560 [details] [review] Bug 20384: unit test for get_all_biblios_iterator & get_all_authorities_iterator
Created attachment 89727 [details] [review] Bug 20384 - Elastic rebuild script improvements : option --file To test: - after testing offset and length, apply this patch - launch perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --file your_file.csv your file need to contain a column of biblionumber from biblio or authid from auth_header (only one column) You can go in Koha -> reports -> Create from sql SELECT biblionumber FROM biblio ; and then download in csv to have your csv file. - You can test with others query like : select biblionumber from biblio WHERE author LIKE 'a%'; Total number of results (411); - verify that you have the rigth number of records indexed
Created attachment 89886 [details] [review] Bug 20384 - Elastic rebuild script improvements - options --offset, --length & --file This patch add the options --offset, --length & --file to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help Option --file : To test: - after testing offset and length, apply this patch - launch perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --file your_file.csv your file need to contain a column of biblionumber from biblio or authid from auth_header (only one column) You can go in Koha -> reports -> Create from sql SELECT biblionumber FROM biblio ; and then download in csv to have your csv file. - You can test with others query like : select biblionumber from biblio WHERE author LIKE 'a%'; Total number of results (411); - verify that you have the rigth number of records indexed
Created attachment 89887 [details] [review] Bug 20384: unit test for get_all_biblios_iterator & get_all_authorities_iterator
Created attachment 89888 [details] [review] Bug 20384 - Elastic rebuild script improvements - options --offset, --length & --file This patch add the options --offset, --length & --file to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help Option --file : To test: - after testing offset and length, apply this patch - launch perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --file your_file.csv your file need to contain a column of biblionumber from biblio or authid from auth_header (only one column) You can go in Koha -> reports -> Create from sql SELECT biblionumber FROM biblio ; and then download in csv to have your csv file. - You can test with others query like : select biblionumber from biblio WHERE author LIKE 'a%'; Total number of results (411); - verify that you have the rigth number of records indexed
Please rebase, does not apply.
Created attachment 106587 [details] [review] Bug 20384: unit test for get_all_biblios_iterator & get_all_authorities_iterator
Created attachment 106588 [details] [review] Bug 20384 - Elastic rebuild script improvements - options --offset, --length & --file This patch add the options --offset, --length & --file to the script: misc/search_tools/rebuild_elastic_search.pl To test : - apply the patch - verify that the script by default is working, launch : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 - launch with the option --offset - then with the option --length - launch with both options : perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --offset 1000 --length 200 - verify that the total records indexed is 200 launch perl misc/search_tools/rebuild_elastic_search.pl --man for help Option --file : To test: - after testing offset and length, apply this patch - launch perl misc/search_tools/rebuild_elastic_search.pl -v -d -c 100 --file your_file.csv your file need to contain a column of biblionumber from biblio or authid from auth_header (only one column) You can go in Koha -> reports -> Create from sql SELECT biblionumber FROM biblio ; and then download in csv to have your csv file. - You can test with others query like : select biblionumber from biblio WHERE author LIKE 'a%'; Total number of results (411); - verify that you have the rigth number of records indexed