Bug 12948 - The language filter in advance search doen't work with DOM MARC21 - NORMARC
Summary: The language filter in advance search doen't work with DOM MARC21 - NORMARC
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching (show other bugs)
Version: master
Hardware: All All
: P5 - low minor (vote)
Assignee: Zeno Tajoli
QA Contact: Testopia
URL:
Keywords:
: 13725 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-09-17 18:39 UTC by Zeno Tajoli
Modified: 2015-12-03 22:02 UTC (History)
4 users (show)

See Also:
Change sponsored?: Sponsored
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Patch for MARC21 (3.36 KB, patch)
2015-02-15 16:59 UTC, Zeno Tajoli
Details | Diff | Splinter Review
Patch for Normarc (2.99 KB, patch)
2015-02-15 17:01 UTC, Zeno Tajoli
Details | Diff | Splinter Review
Bug 12948 - [SIGNED-OFF] Patch-for-filter-language-in-Advanced-Search (3.64 KB, patch)
2015-02-16 09:35 UTC, Magnus Enger
Details | Diff | Splinter Review
Bug 12948 - [SIGNED-OFF] Patch for filter 'language' in Advanced Search. (3.67 KB, patch)
2015-02-16 09:39 UTC, Magnus Enger
Details | Diff | Splinter Review
[SIGNED OFF] Bug 12948: Use word indexing for language (NORMARC) (3.69 KB, patch)
2015-02-18 12:58 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
[SIGNED OFF] Bug 12948: Use word indexing for language (MARC21) (3.71 KB, patch)
2015-02-18 12:58 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
[PASSED QA] Bug 12948: Use word indexing for language (NORMARC) (3.72 KB, patch)
2015-02-20 14:47 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 12948: Use word indexing for language (MARC21) (3.75 KB, patch)
2015-02-20 14:49 UTC, Kyle M Hall
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Zeno Tajoli 2014-09-17 18:39:23 UTC
The filter 'language' in Advanced Search works on 041 data and not with data in 008 pos 35-28.

I think there is an 'ln:n' instead of 'ln:w' in specific lines of 
marc21/biblios/biblio-koha-indexdefs.xml.

This bug is present only on DOM indexing setup for MARC21 and probably in NORMARC.

The Unimarc equivalent is OK
Comment 1 Magnus Enger 2015-02-12 21:43:08 UTC
Zeno: You are right, ln:n needs to be replaced with ln:w, both for MARC21 and NORMARC. I can do a patch, but now that grs1 is on the way out, do we still edit biblio-koha-indexdefs.xml and generate biblio-zebra-indexdefs.xsl from that, or do we edit biblio-zebra-indexdefs.xsl directly?
Comment 2 Zeno Tajoli 2015-02-13 10:24:50 UTC
Hi,

I think that the best option now is  to edit biblio-koha-indexdefs.xml and generate biblio-zebra-indexdefs.xsl from that.

But I ask about this topic in koha-dev: http://lists.koha-community.org/pipermail/koha-devel/2015-February/041178.html

Bye
Comment 3 Magnus Enger 2015-02-13 10:32:05 UTC
Thanks! Feel free to do the patch if you want. :-)
Comment 4 Zeno Tajoli 2015-02-15 16:59:04 UTC Comment hidden (obsolete)
Comment 5 Zeno Tajoli 2015-02-15 17:01:06 UTC Comment hidden (obsolete)
Comment 6 Zeno Tajoli 2015-02-15 17:04:10 UTC
To test on MARC21: 
1)Setup a site with
 MARC21 
2)Insert 2 record, one lang A in 041 and 008 pos
 35-37 an other with lang A in 041 and lang B in 008 pos
 35-37 
3)Index them 
4)Search in advanced search with filter
 'languare' for lan A. You will see 2 records 
5)Search in
 advanced search with filter 'languare' for lan B. You will
 see 0 records 
6)Apply the patch 
7)Full reindex
8)Search in advanced search
 with filter 'languare' for lan B. You will see 1 records

To test on Normarc: the same as MARC21, but you need a Normarc setup
Comment 7 Magnus Enger 2015-02-16 08:03:57 UTC
(In reply to Zeno Tajoli from comment #2)
> Hi,
> 
> I think that the best option now is  to edit biblio-koha-indexdefs.xml and
> generate biblio-zebra-indexdefs.xsl from that.
> 
> But I ask about this topic in koha-dev:
> http://lists.koha-community.org/pipermail/koha-devel/2015-February/041178.
> html

Tomas answered this on the mailing list, I'm pasting in the answer here for future reference:

---

The existence of biblio-koha-indexdefs.xml is not related to GRS-1 right now. It might have been (for a while) a way to transform the record.abs file into something translatable to XSLT. But it is a more useful thing now.

The XML file provides with a user-friendly syntax for defining indexes (and now also facets). The etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl file is designed to process that syntax for creating a suitable XSLT for use in DOM.

So, if you want to change some indexes, or facet definitions: do it on the XML file.

If you want to change the behaviour of DOM indexing (for example, you could want to generate an index for the 'availability' filter on each record, so it is calculated on indexing time instead of complex un-doable PQF queries), then you should patch the meta-XSLT (koha-indexdefs-to-zebra.xsl) so it generates an XSLT that generates that specific index out of a MARCXML record.

It's been a while since we stopped re-generating the DOM files from the GRS-1, as we've been forcing people to fix both GRS-1 *and* DOM each time.

Hope this clarifies the landscape.
Regards
Tomas

http://lists.koha-community.org/pipermail/koha-devel/2015-February/041179.html
Comment 8 Magnus Enger 2015-02-16 09:35:46 UTC Comment hidden (obsolete)
Comment 9 Magnus Enger 2015-02-16 09:39:28 UTC Comment hidden (obsolete)
Comment 10 Tomás Cohen Arazi 2015-02-18 12:42:22 UTC
*** Bug 13725 has been marked as a duplicate of this bug. ***
Comment 11 Tomás Cohen Arazi 2015-02-18 12:58:47 UTC Comment hidden (obsolete)
Comment 12 Tomás Cohen Arazi 2015-02-18 12:58:57 UTC Comment hidden (obsolete)
Comment 13 Kyle M Hall 2015-02-20 14:47:13 UTC
Created attachment 36090 [details] [review]
[PASSED QA] Bug 12948: Use word indexing for language (NORMARC)

This patch is for Normarc
Same test plan as patch for MARC21, except you need a setup with Normarc.

http://bugs.koha-community.org/show_bug.cgi?id=12948
Signed-off-by: Magnus Enger <magnus@enger.priv.no>

- Added a record with "bul" in 008pos35-37
- Verified that this did not turn up in an advanced search with language =
  Bulgarian
- Applied the patch
- I was testing on a gitified install, so I had to copy the patched index file
  to the right location with this command:

sudo cp etc/zebradb/marc_defs/normarc/biblios/biblio-zebra-indexdefs.xsl \
/etc/koha/zebradb/marc_defs/normarc/biblios/biblio-zebra-indexdefs.xsl

- Did a full reindex
- Verified that the record *did* turn up in an advanced search with language =
  Bulgarian
- Signing off! Thanks Zeno!

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 14 Kyle M Hall 2015-02-20 14:49:52 UTC
Created attachment 36091 [details] [review]
[PASSED QA] Bug 12948: Use word indexing for language (MARC21)

 This patch is for MARC21. To test:
1)Setup a site with
 MARC21
2)Insert 2 record, one lang A in 041 and 008 pos
 35-37 an other with lang A in 041 and lang B in 008 pos
 35-37
3)Index them
4)Search in advanced search with filter
 'languare' for lan A. You will see 2 records
5)Search in
 advanced search with filter 'languare' for lan B. You will
 see 0 records
6)Apply the patch
7)Full reindex
8)Search in advanced search
 with filter 'languare' for lan B. You will see 1 records

http://bugs.koha-community.org/show_bug.cgi?id=12948

Signed-off-by: Magnus Enger <magnus@enger.priv.no>
I have *not* actually tested this, but the changes are identical to the ones
done for NORMARC, which I have tested, so I think it is safe to sign off. If
anyone disagrees, please reset the bug to "Needs signoff".

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 15 Tomás Cohen Arazi 2015-02-20 14:53:50 UTC
Patches pushed to master.

Thanks Zeno!
Comment 16 Chris Cormack 2015-02-22 23:16:09 UTC
Pushed to 3.18.x will be in 3.18.4