Bug 12948

Summary: The language filter in advance search doen't work with DOM MARC21 - NORMARC
Product: Koha Reporter: Zeno Tajoli <ztajoli>
Component: SearchingAssignee: Zeno Tajoli <ztajoli>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: minor    
Priority: P5 - low CC: chris, januszop, magnus, tomascohen
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: Sponsored Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Patch for MARC21
Patch for Normarc
Bug 12948 - [SIGNED-OFF] Patch-for-filter-language-in-Advanced-Search
Bug 12948 - [SIGNED-OFF] Patch for filter 'language' in Advanced Search.
[SIGNED OFF] Bug 12948: Use word indexing for language (NORMARC)
[SIGNED OFF] Bug 12948: Use word indexing for language (MARC21)
[PASSED QA] Bug 12948: Use word indexing for language (NORMARC)
[PASSED QA] Bug 12948: Use word indexing for language (MARC21)

Description Zeno Tajoli 2014-09-17 18:39:23 UTC
The filter 'language' in Advanced Search works on 041 data and not with data in 008 pos 35-28.

I think there is an 'ln:n' instead of 'ln:w' in specific lines of 
marc21/biblios/biblio-koha-indexdefs.xml.

This bug is present only on DOM indexing setup for MARC21 and probably in NORMARC.

The Unimarc equivalent is OK
Comment 1 Magnus Enger 2015-02-12 21:43:08 UTC
Zeno: You are right, ln:n needs to be replaced with ln:w, both for MARC21 and NORMARC. I can do a patch, but now that grs1 is on the way out, do we still edit biblio-koha-indexdefs.xml and generate biblio-zebra-indexdefs.xsl from that, or do we edit biblio-zebra-indexdefs.xsl directly?
Comment 2 Zeno Tajoli 2015-02-13 10:24:50 UTC
Hi,

I think that the best option now is  to edit biblio-koha-indexdefs.xml and generate biblio-zebra-indexdefs.xsl from that.

But I ask about this topic in koha-dev: http://lists.koha-community.org/pipermail/koha-devel/2015-February/041178.html

Bye
Comment 3 Magnus Enger 2015-02-13 10:32:05 UTC
Thanks! Feel free to do the patch if you want. :-)
Comment 4 Zeno Tajoli 2015-02-15 16:59:04 UTC Comment hidden (obsolete)
Comment 5 Zeno Tajoli 2015-02-15 17:01:06 UTC Comment hidden (obsolete)
Comment 6 Zeno Tajoli 2015-02-15 17:04:10 UTC
To test on MARC21: 
1)Setup a site with
 MARC21 
2)Insert 2 record, one lang A in 041 and 008 pos
 35-37 an other with lang A in 041 and lang B in 008 pos
 35-37 
3)Index them 
4)Search in advanced search with filter
 'languare' for lan A. You will see 2 records 
5)Search in
 advanced search with filter 'languare' for lan B. You will
 see 0 records 
6)Apply the patch 
7)Full reindex
8)Search in advanced search
 with filter 'languare' for lan B. You will see 1 records

To test on Normarc: the same as MARC21, but you need a Normarc setup
Comment 7 Magnus Enger 2015-02-16 08:03:57 UTC
(In reply to Zeno Tajoli from comment #2)
> Hi,
> 
> I think that the best option now is  to edit biblio-koha-indexdefs.xml and
> generate biblio-zebra-indexdefs.xsl from that.
> 
> But I ask about this topic in koha-dev:
> http://lists.koha-community.org/pipermail/koha-devel/2015-February/041178.
> html

Tomas answered this on the mailing list, I'm pasting in the answer here for future reference:

---

The existence of biblio-koha-indexdefs.xml is not related to GRS-1 right now. It might have been (for a while) a way to transform the record.abs file into something translatable to XSLT. But it is a more useful thing now.

The XML file provides with a user-friendly syntax for defining indexes (and now also facets). The etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl file is designed to process that syntax for creating a suitable XSLT for use in DOM.

So, if you want to change some indexes, or facet definitions: do it on the XML file.

If you want to change the behaviour of DOM indexing (for example, you could want to generate an index for the 'availability' filter on each record, so it is calculated on indexing time instead of complex un-doable PQF queries), then you should patch the meta-XSLT (koha-indexdefs-to-zebra.xsl) so it generates an XSLT that generates that specific index out of a MARCXML record.

It's been a while since we stopped re-generating the DOM files from the GRS-1, as we've been forcing people to fix both GRS-1 *and* DOM each time.

Hope this clarifies the landscape.
Regards
Tomas

http://lists.koha-community.org/pipermail/koha-devel/2015-February/041179.html
Comment 8 Magnus Enger 2015-02-16 09:35:46 UTC Comment hidden (obsolete)
Comment 9 Magnus Enger 2015-02-16 09:39:28 UTC Comment hidden (obsolete)
Comment 10 Tomás Cohen Arazi 2015-02-18 12:42:22 UTC
*** Bug 13725 has been marked as a duplicate of this bug. ***
Comment 11 Tomás Cohen Arazi 2015-02-18 12:58:47 UTC Comment hidden (obsolete)
Comment 12 Tomás Cohen Arazi 2015-02-18 12:58:57 UTC Comment hidden (obsolete)
Comment 13 Kyle M Hall 2015-02-20 14:47:13 UTC
Created attachment 36090 [details] [review]
[PASSED QA] Bug 12948: Use word indexing for language (NORMARC)

This patch is for Normarc
Same test plan as patch for MARC21, except you need a setup with Normarc.

http://bugs.koha-community.org/show_bug.cgi?id=12948
Signed-off-by: Magnus Enger <magnus@enger.priv.no>

- Added a record with "bul" in 008pos35-37
- Verified that this did not turn up in an advanced search with language =
  Bulgarian
- Applied the patch
- I was testing on a gitified install, so I had to copy the patched index file
  to the right location with this command:

sudo cp etc/zebradb/marc_defs/normarc/biblios/biblio-zebra-indexdefs.xsl \
/etc/koha/zebradb/marc_defs/normarc/biblios/biblio-zebra-indexdefs.xsl

- Did a full reindex
- Verified that the record *did* turn up in an advanced search with language =
  Bulgarian
- Signing off! Thanks Zeno!

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 14 Kyle M Hall 2015-02-20 14:49:52 UTC
Created attachment 36091 [details] [review]
[PASSED QA] Bug 12948: Use word indexing for language (MARC21)

 This patch is for MARC21. To test:
1)Setup a site with
 MARC21
2)Insert 2 record, one lang A in 041 and 008 pos
 35-37 an other with lang A in 041 and lang B in 008 pos
 35-37
3)Index them
4)Search in advanced search with filter
 'languare' for lan A. You will see 2 records
5)Search in
 advanced search with filter 'languare' for lan B. You will
 see 0 records
6)Apply the patch
7)Full reindex
8)Search in advanced search
 with filter 'languare' for lan B. You will see 1 records

http://bugs.koha-community.org/show_bug.cgi?id=12948

Signed-off-by: Magnus Enger <magnus@enger.priv.no>
I have *not* actually tested this, but the changes are identical to the ones
done for NORMARC, which I have tested, so I think it is safe to sign off. If
anyone disagrees, please reset the bug to "Needs signoff".

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 15 Tomás Cohen Arazi 2015-02-20 14:53:50 UTC
Patches pushed to master.

Thanks Zeno!
Comment 16 Chris Cormack 2015-02-22 23:16:09 UTC
Pushed to 3.18.x will be in 3.18.4