Bug 9830 - Some UNIMARC item indexes could be broken whith Queryweightfields syspref
Summary: Some UNIMARC item indexes could be broken whith Queryweightfields syspref
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Mathieu Saby
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-03-17 14:31 UTC by Mathieu Saby
Modified: 2014-12-07 20:07 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
[PATCH] Bug 9830: Fix some indexes in UNIMARC item indexing (11.25 KB, patch)
2013-03-17 14:55 UTC, Mathieu Saby
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 9830: Fix some indexes in UNIMARC item indexing (11.68 KB, patch)
2013-09-12 21:55 UTC, Bernardo Gonzalez Kriegel
Details | Diff | Splinter Review
[PATCH] [SIGNED-OFF] Bug 9830: Fix some indexes in UNIMARC item indexing (12.02 KB, patch)
2013-10-11 22:13 UTC, Mathieu Saby
Details | Diff | Splinter Review
Bug 9830: Fix some indexes in UNIMARC item indexing (12.07 KB, patch)
2013-10-21 12:49 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Mathieu Saby 2013-03-17 14:31:08 UTC
With this combinaison of sysprefs, and a UNIMARC configuration, it impossible to search on location, barcode and ccode indexes :

QueryWeightFields          is activated
QueryAutoTruncate          only if * is added

In UNIMARC, location, barcode and ccode (995 $e,$f,h) are indexed only as "words". They need to be indexed also as "phrase"
Additionnaly, in UNIMARC, information about damaged and lost status of items is not indexed, while it is done in MARC21.

M. Saby
Comment 1 Mathieu Saby 2013-03-17 14:55:29 UTC Comment hidden (obsolete)
Comment 2 Bernardo Gonzalez Kriegel 2013-09-12 21:55:24 UTC Comment hidden (obsolete)
Comment 3 Mathieu Saby 2013-09-12 22:31:19 UTC
I suppose it was a child's play for you to test after bug 8252 ;-)

Mathieu
Comment 4 Bernardo Gonzalez Kriegel 2013-09-12 22:36:35 UTC
(In reply to mathieu saby from comment #3)
> I suppose it was a child's play for you to test after bug 8252 ;-)
> 
> Mathieu

Indeed!
Now I'm after bug 10037 :)

Bernardo
Comment 5 Vitor Fernandes 2013-09-13 09:03:34 UTC
This also happens for item call numbers.
Besides of that, call number ordering on OPAC or Backoffice results doesn't work at all.
Comment 6 Vitor Fernandes 2013-09-13 09:29:28 UTC
Turning off Queryweightfields, I can search for barcodes and callnumbers in keyword searches (kw,wrdl). Turning on Queryweightfields, I get always 0 results.

I've done the changes on the patch for barcodes, and I still get 0 results with Queryweightfields on and after a complete reindex.
Comment 7 Vitor Fernandes 2013-09-13 09:41:07 UTC
I think the problem is with QueryAutoTruncate.
My last comment was incorrect.

I've done the change on records.abs and xml files to add barcode:p to 995$f.
Then I've done a full reindex.

With Queryweightfields on and QueryAutoTruncate set to "only if * is added", a search for a barcode (example 01000039875) gives 0 results. If I put a * in the end of the search (exemple 01000039875*), I get the record with an item with the barcode.
If QueryAutoTruncate is set to automatically I get also the record.

With callnumbers happens the same thing.
Besides of that callnumber ordering isn't working, maybe because of this problem.
Comment 8 Mathieu Saby 2013-09-13 11:24:30 UTC
Vitor, I will take a look tomorrow.
Note that this patch does not change the indexation of callnumbers, only barcodes, ccodes and location.

M. Saby
Comment 9 Mathieu Saby 2013-09-15 17:11:29 UTC
Vitor, after applying the patch, did you copy the modified files biblio-*-indexdefs.xml and record.abs to etc/zebra in your main Koha repository? I forget to mention that, but without that step it won't work.

For callnumbers, if the pbm is persistant, I will try to write an other patch. Let's push this one if it is working what it is meant for.

M. Saby
Comment 10 Vitor Fernandes 2013-09-16 09:21:50 UTC
Hi Mathieu,

I didn't copy the files, but I've changed them to test barcode:w indexation.
But didn't worked...
Comment 11 Jonathan Druart 2013-10-10 11:44:49 UTC
Mathieu, the commit message says the patch adds 2 indexes: damaged and lost. But in fact it adds damaged and withdrawn.

Could you fix the commit message please?
Marked as Failed QA.
Comment 12 Katrin Fischer 2013-10-11 16:00:08 UTC
Hi Jonathan, I think fixing that in the commit message when doing QA would be ok - I also tend to fix typos where I spot them :)
Comment 13 Mathieu Saby 2013-10-11 22:13:23 UTC Comment hidden (obsolete)
Comment 14 Mathieu Saby 2013-10-11 22:15:46 UTC
Note : I made the test on current master. the 2 indexes damaged and withdrawn are working well (with DOM). lost index is also working well, like before.

Mathieu
Comment 15 Mathieu Saby 2013-10-11 22:30:52 UTC
By the way, Jonathan, the idea for this patch comes from Fridolyn (MT10271).

Mathieu
Comment 16 Jonathan Druart 2013-10-21 12:49:06 UTC
Created attachment 22154 [details] [review]
Bug 9830: Fix some indexes in UNIMARC item indexing

[typo fixed in commit message ; steps 6 and 7 added in test plan]

With this combination of sysprefs, and a UNIMARC configuration, it impossible to search on location, barcode and ccode indexes :

QueryWeightFields          is activated
QueryAutoTruncate          only if * is added

But in UNIMARC, location, barcode and ccode (995 $e,$f,h) are indexed only as "words". They need to be indexed also as "phrase"
Additionnaly, in UNIMARC, information about damaged and withdrawn status of items is not indexed, while it is done in MARC21.
This patch
- add 2 new indexes for 995$1 (damaged) and 995$3 (withdrawn)
- index location, barcode and ccode as "phrase" as well as "words"
indexing of items in UNIMARC could be improved later. So this patch also add comments explaining the origin of Koha 995, I think it could be usefull for further changes.

To test, on a UNIMARC configuration :
A. indexed with GRS-1
1) set sysprefs QueryWeightFields as "activated" and QueryAutoTruncate as "only if * is added"
2) select location index in advanced search and search for a value existing in your records in 995$e => 0 results
3) Apply patch
4) Rebuild zebra
5) select location index in advanced search and search for a value existing in your records in 995$e => x results
6) mark an item as withdrawn ; search "withdrawn:1" => x results, and among them the biblio to which the item is attached
7) mark an item as damaged ; search "damaged:1" => x results, and among them the biblio to which the item is attached

B. indexed with DOM
Do the same operations

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. No koha-qa errors

Test
Apply the patch
Begin with GRS-1
Full reindex
Search by location, no results
cp files biblio-*-indexdefs.xml and record.abs to destination on etc/zebra
Full reindex
Search by location, got results

Switch to DOM
reset files
Full reindex
Search by location, no results
cp files
Full reindex
Search by location, results !

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Comment 17 Galen Charlton 2013-10-21 15:41:21 UTC
Pushed to master.  Thanks, Mathieu!