The upgrade from 1.X to 5.X works very well for indexing existing records, however, it seems the '_id' field is not being given to elastic at save so it is generating a random id. The problem with this is that each record update will save a new record as opposed to overwriting the existing record and you will get multiple items in your search. To reproduce: 1 - Have ES5 running and have some records 2 - Choose a biblionumber and reindex just that record, deleting the index before: perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d -b -bn 2365 3 - View that record on the command line: curl -XGET 'localhost:9200/koha_kohadev_biblios/_search/?pretty=true&q=es_id:2365' 4 - Note the '_id' field is of the form: "_id" : "AVoemHc9LtgvIQ1yOOU6", 5 - Search for the record in koha - should be one 6 - Add or edit the items on that record 7 - Search again, the record returns multiple times This should be a small fix, not sure where the disconnect is though
Olli, any chances you can try and fix this one?
Sorry. Busy. I (or somebody in Koha-Suomi) will solve the ES issues when we decide to go to ES. Can't tell from the top of my hat, where the problem is. Sounds like something to do with interfacing via Catmandu. Catmandu needs some special care to handle _id properly. Maybe the issue is with one of the multiple Catmandu modules needed to get the MARC::Record to ES. Do we really need Catmandu at this phase? We can directly index to ES, or any search engine because Koha already has the infrastucture to facilitate multiple search engines. Now we need to wait for the Catmandu-guys to port Catmandu::Store::ElasticSearch to 5.2 or supply a patch (if the problem is in Catmandu) Also We need to implement Catmandu in the JavaScript-side if we want to consume the results directly to the browser. We would have to implement any parser there though. AJAX-searches would be nice! I guess Catmandu defines some kind of a "standard" for working with library data, but so does MARCXML. Turning MARCXML to JSON is really simple. Or even better... JSON-LD!! But this is not an answer to the bug at hand :) My bet is Catmandu. Giving it the right parameters to handle _id.
Nvm - this is a kohadevbox issue - version of Catmandu installed is not the .deb from bug 17255 was using version 3.03 - need .5 or greater
Ran into this too. For the record: ES log says "MapperParsingException[Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters.]" You need to cpan Catmandu::Store::ElasticSearch, not Catmandu::Store::Elasticsearch S vs s!