Bug 6172

Summary: Non-French UNIMARC frameworks need to be updated
Product: Koha Reporter: Serhij Dubyk <dubyk>
Component: SearchingAssignee: Galen Charlton <gmcharlt>
Status: CLOSED FIXED QA Contact: Bugs List <koha-bugs>
Severity: normal    
Priority: P5 - low CC: dubyk, f.demians, fridolin.somers, giuseppe.ciaccio, J.P.Knight, koha, mathsabypro
Version: Main   
Hardware: All   
OS: Linux   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18309
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10726
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Bug Depends on: 11453    
Bug Blocks:    
Attachments: simple entered biblio

Description Serhij Dubyk 2011-04-12 10:39:10 UTC
Created attachment 3888 [details]
simple entered biblio

In general: not searched records entered in the Unimarc-environment.

Used Debian packages of Koha (from squeeze and squeeze-dev). Currently it is: koha-common_3.2.6-1_all.deb or koha-common_3.3-1~git20110407105508.b1678665_all.deb. Tried install in current stable Debian and unstable variants.

In web-installer was choosed English Unimarc (similar result for Ukrainian and Russian Unimarc).

After install created simple biblio-record (with title=Book) and try to reindex zebra with next warnings/errors:

{
#>sudo KOHA_CONF=/etc/koha/sites/ukr_unimarc/koha-conf.xml PERL5LIB=/usr/share/koha/lib /usr/share/koha/bin/migration_tools/rebuild_zebra.pl -b -r -v -x -k -nosanitize

Zebra configuration information
================================
Zebra biblio directory      = /var/lib/koha/ukr_unimarc/biblios
Zebra authorities directory = /var/lib/koha/ukr_unimarc/authorities
Koha directory              = /usr/share/koha/intranet/cgi-bin
BIBLIONUMBER in :     090$9
BIBLIOITEMNUMBER in : 090$a
================================
skipping authorities
====================
exporting biblio
====================
1
Records exported: 1
====================
REINDEXING zebra
====================
10:27:12-12/04 zebraidx(4846) [log] zebra_start 2.0.44 419ad759807269fdfa379799a051ed3a551c6541
10:27:12-12/04 zebraidx(4846) [log] config /etc/koha/sites/ukr_unimarc/zebra-biblios.cfg
10:27:12-12/04 zebraidx(4846) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-alvis.so
10:27:12-12/04 zebraidx(4846) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-grs-marc.so
10:27:12-12/04 zebraidx(4846) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-dom.so
10:27:12-12/04 zebraidx(4846) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-text.so
10:27:12-12/04 zebraidx(4846) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-grs-regx.so
10:27:12-12/04 zebraidx(4846) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-grs-xml.so
10:27:12-12/04 zebraidx(4846) [log] enabling shadow spec=/var/lib/koha/ukr_unimarc/biblios/shadow:4G
10:27:12-12/04 zebraidx(4846) [log] cache_fname = /var/lib/koha/ukr_unimarc/biblios/shadow/cache
10:27:12-12/04 zebraidx(4846) [log] zebra_stop: 0.06 0.00 0.00
10:27:12-12/04 zebraidx(4847) [log] zebra_start 2.0.44 419ad759807269fdfa379799a051ed3a551c6541
10:27:12-12/04 zebraidx(4847) [log] config /etc/koha/sites/ukr_unimarc/zebra-biblios.cfg
10:27:12-12/04 zebraidx(4847) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-alvis.so
10:27:12-12/04 zebraidx(4847) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-grs-marc.so
10:27:12-12/04 zebraidx(4847) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-dom.so
10:27:12-12/04 zebraidx(4847) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-text.so
10:27:12-12/04 zebraidx(4847) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-grs-regx.so
10:27:12-12/04 zebraidx(4847) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-grs-xml.so
10:27:12-12/04 zebraidx(4847) [log] enabling shadow spec=/var/lib/koha/ukr_unimarc/biblios/shadow:4G
10:27:12-12/04 zebraidx(4847) [log] cache_fname = /var/lib/koha/ukr_unimarc/biblios/shadow/cache
10:27:13-12/04 zebraidx(4847) [log] dir /tmp/Bjp22Rn17_/biblio
10:27:13-12/04 zebraidx(4847) [warn] Unknown register type: 0
10:27:13-12/04 zebraidx(4847) [warn] Record didn't contain match fields in (bib1,Local-number)
10:27:13-12/04 zebraidx(4847) [log] error grs.sgml /tmp/Bjp22Rn17_/biblio/exported_records 0
10:27:13-12/04 zebraidx(4847) [log] Iterations: isam/dict 64/19
10:27:13-12/04 zebraidx(4847) [log] Dict: inserts/updates/deletions: 19/0/0
10:27:13-12/04 zebraidx(4847) [log] Records: 0 i/u/d 0/0/0
10:27:13-12/04 zebraidx(4847) [log] zebra_stop: 0.14 0.02 0.00
10:27:13-12/04 zebraidx(4850) [log] zebra_start 2.0.44 419ad759807269fdfa379799a051ed3a551c6541
10:27:13-12/04 zebraidx(4850) [log] config /etc/koha/sites/ukr_unimarc/zebra-biblios.cfg
10:27:13-12/04 zebraidx(4850) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-alvis.so
10:27:13-12/04 zebraidx(4850) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-grs-marc.so
10:27:13-12/04 zebraidx(4850) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-dom.so
10:27:13-12/04 zebraidx(4850) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-text.so
10:27:13-12/04 zebraidx(4850) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-grs-regx.so
10:27:13-12/04 zebraidx(4850) [log] Loaded filter module /usr/lib/idzebra-2.0/modules/mod-grs-xml.so
10:27:13-12/04 zebraidx(4850) [log] enabling shadow spec=/var/lib/koha/ukr_unimarc/biblios/shadow:4G
10:27:13-12/04 zebraidx(4850) [log] cache_fname = /var/lib/koha/ukr_unimarc/biblios/shadow/cache
10:27:13-12/04 zebraidx(4850) [log] zebra_stop: 0.05 0.00 0.00
====================
CLEANING
}

In result in Koha this record not searched.

(also tried with "koha-rebuil-zebra ukr_unimarc" - not present any output but the result was the same - record not searched)

Key-warning:
[warn] Record didn't contain match fields in (bib1,Local-number)

This warning is present for any instances with Unimarc.

found in next files:

1) /etc/koha/zebradb/marc_defs/unimarc/biblios/record.abs:
...
melm 090$9      Local-number,Local-number:n
...

2) /etc/koha/zebradb/biblios/etc/bib1.att
...
att 12    Local-number
...

3) /etc/koha/sites/ukr_unimarc/zebra-biblios.cfg
...
recordId: (bib1,Local-number)
...

What's interesting:
In marc21 zebra-indexing work without warnings about "Local-number" and similar entered biblio was searched.
In marc21-file /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs
this parameter commented:
...
#melm 035       Local-number,Identifier-standard
#melm 090$c     Local-number:w
...

I was tried also comment this in /etc/koha/zebradb/marc_defs/unimarc/biblios/record.abs with no result

In zebra logs not found any errors/warnings during this indexing.

I don't understand what's wrong with Unimarc and Zebra...
Comment 1 Frédéric Demians 2011-05-06 06:07:01 UTC
French UNIMARC works, and is used and maintained for a lot of French
libraries. Other languages UNIMARC frameworks are a kind of 'fork' of
the French version. It is an orphaned fork that no one is actively
maintaining. So your bug means that French UNIMARC framework has shifted
and is now different from the other ones.

A quick look, shows that Zebra indexes definition file (record.abs)
expects Koka biblio record ID (biblionumber) to be in 090$a field when
in English UNIMARC framework biblionumber is in 001. You should modify
by hand your framework in order to solve your issue.
Comment 2 Katrin Fischer 2012-03-19 06:27:28 UTC
Reading Frederic's comment it's perhaps worth investigating the sample frameworks in Koha are correct for UNIMARC, so we can make it a little easier for people installing in other languages.
Comment 3 Fridolin Somers 2012-11-15 09:39:58 UTC
> BIBLIONUMBER in :     090$9
This shows that in your framework, biblionumber is mapped to 090$9.

> melm 090$9      Local-number,Local-number:n
This is good.
But are there other fields indexed with Local-number is your records.abs ?
Comment 4 Mathieu Saby 2013-12-07 14:07:19 UTC
Non french UNIMARC frameworks should be aligned on "IFLA UNIMARC"

Mathieu
Comment 5 Katrin Fischer 2014-03-15 09:50:16 UTC
Linking this to 11453, as updating the English UNIMARC framework is a first step on making it possible to generate the frameworks in different languages using the translation tools.

*** This bug has been marked as a duplicate of bug 11453 ***
Comment 6 Katrin Fischer 2014-03-15 09:51:10 UTC
Linking this to 11453, as updating the English UNIMARC framework is a first step on making it possible to generate the frameworks in different languages using the translation tools.
Comment 7 Katrin Fischer 2014-11-03 22:26:45 UTC
We have an updated English UNIMARC framework now with 11453. I am not sure it makes sense to continue here - maybe we should file more specific bugs for further improvements/fixes?
Comment 8 Jon Knight 2018-05-02 10:10:25 UTC
As this bug is marked as a duplicate of bug 11453, which is now "CLOSED FIXED", can this bug also be marked as closed?  Not being a UNIMARC user I'm not up on the nuances of French/Non-French frameworks for it, but I'm just tidying up reopened cases with my "Bug Wrangler" hat on.

If nobody responds in a week or two, I'll assume it is ready to be closed.
Comment 9 Giuseppe Ciaccio 2018-05-09 07:43:29 UTC
Please don't close yet.  I have been trying installing Koha 17.11 since a month ago, the unimarc flavour but english lang, and nothing works yet, with same malfunction as described in this bug report.  I plan to do tests with other languages and report here.  Thank you.
Comment 10 Jon Knight 2018-05-09 08:45:54 UTC
(In reply to Giuseppe Ciaccio from comment #9)
> Please don't close yet.  I have been trying installing Koha 17.11 since a
> month ago, the unimarc flavour but english lang, and nothing works yet, with
> same malfunction as described in this bug report.  I plan to do tests with
> other languages and report here.  Thank you.

OK, I'll leave it alone! :-)
Comment 11 Giuseppe Ciaccio 2018-05-10 15:45:24 UTC
It seems that Koha/unimarc/en is working properly, provided the indexer has started (which does not happen initially, but this is another story; see https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20745).
Comment 12 Jon Knight 2018-05-10 15:59:16 UTC
(In reply to Giuseppe Ciaccio from comment #11)
> It seems that Koha/unimarc/en is working properly, provided the indexer has
> started (which does not happen initially, but this is another story; see
> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20745).

As the Koha/unimarc/en is working OK, I assume you're happy for this case to close?
Comment 13 Giuseppe Ciaccio 2018-05-10 16:09:48 UTC
Yes I'm indeed :)
Comment 14 Katrin Fischer 2019-04-28 23:12:52 UTC
(In reply to Jon Knight from comment #12)
> (In reply to Giuseppe Ciaccio from comment #11)
> > It seems that Koha/unimarc/en is working properly, provided the indexer has
> > started (which does not happen initially, but this is another story; see
> > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20745).
> 
> As the Koha/unimarc/en is working OK, I assume you're happy for this case to
> close?

(In reply to Giuseppe Ciaccio from comment #13)
> Yes I'm indeed :)