Bug 22430 - importing from a UNIMARC backup will import biblio records but not items
Summary: importing from a UNIMARC backup will import biblio records but not items
Status: CLOSED INVALID
Alias: None
Product: Koha
Classification: Unclassified
Component: MARC Bibliographic record staging/import (show other bugs)
Version: 18.11
Hardware: All All
: P1 - high critical with 10 votes (vote)
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-02-28 09:42 UTC by Giuseppe Ciaccio
Modified: 2020-01-06 20:13 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
a test backup containing 10 bibliorecords and 21 items (UNIMARC) (10.55 KB, application/octet-stream)
2019-03-06 20:39 UTC, Giuseppe Ciaccio
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Giuseppe Ciaccio 2019-02-28 09:42:31 UTC
Play the following backup/restore game:

1) create two empty instances, A and B, both in the UNIMARC flavour;
2) populate A with ten bibliorecords, two of which with associated items;
3) create a backup from A so that only UNIMARC records having items are exported;  the backup indeed contains only two records;
4) import the backup into instance B;
5) contrary to expectations, see that B now contains two biblio records but
no items associated to them.

If I inspect the records in the catalog of B, I see that they are void of
the field 995 (https://wiki.koha-community.org/wiki/Holdings_data_fields_%289xx%29#UNIMARC_Holding_field.2C_3.2_default_-_field_995). However, the records staged in the temporary area of B during import
phase do have that field.

So it seems that the UNIMARC item field is correctly fetched when importing the backup in the stage area, but discarded when the staged records are pushed in the catalog.

This bug is dangerous because it prevents the backup/restore procedure to work. Backup/restore is a necessary step to preserve the biblio archive in case of hardware/software failures.  So this bug may cause a permanent loss of the archive.
Comment 1 Jonathan Druart 2019-03-05 16:47:49 UTC
What is a backup in this context?

I have exported the records then imported them in a fresh DB.
I got:
MARC staging results :
    Processing bibliographic records
    1 records in file
    0 records not staged because of MARC error
    1 records staged
    Did not check for matches with existing records in catalog
    5 item records found and staged

then:
MariaDB [koha_kohadev]> select count(*) from items;
+----------+
| count(*) |
+----------+
|        5 |
+----------+

All looks good for me (items are displayed on the detail page of the bibliographic record).

Tested with UNIMARC.
Comment 2 Fridolin Somers 2019-03-06 14:38:53 UTC
(In reply to Giuseppe Ciaccio from comment #0)
> So it seems that the UNIMARC item field is correctly fetched when importing
> the backup in the stage area, but discarded when the staged records are
> pushed in the catalog.

I think I found why.
The push to catalog is not allowed because the items have the same barcode which must be unique in catalog.
Comment 3 Giuseppe Ciaccio 2019-03-06 20:39:32 UTC
Created attachment 86216 [details]
a test backup containing 10 bibliorecords and 21 items (UNIMARC)

Try with these records.  On my koha installs (two ones at the moment, a 17.11 and a 18.11), importing this batch results into 10 records in the catalog but no items.  The batch however contains 21 items, and these are visible in the record copies left in the stage area.
Comment 4 Giuseppe Ciaccio 2019-03-06 20:59:22 UTC
(In reply to Jonathan Druart from comment #1)
> What is a backup in this context?

Backup is: saving data away, and use them to restore a service after a catastrophic failure.  In koha I would export records to a file, and import them again in case the original archive is lost.

> I have exported the records then imported them in a fresh DB.
> I got:
> MARC staging results :
>     Processing bibliographic records
>     1 records in file
>     0 records not staged because of MARC error
>     1 records staged
>     Did not check for matches with existing records in catalog
>     5 item records found and staged
> 
> then:
> MariaDB [koha_kohadev]> select count(*) from items;
> +----------+
> | count(*) |
> +----------+
> |        5 |
> +----------+
> 
> All looks good for me (items are displayed on the detail page of the
> bibliographic record).
> 
> Tested with UNIMARC.

Could you also search the OPAC interface and see the items from there too?

I tried the import of the UNIMARC batch now attached to this bugreport. From the admin interface, Home -> Tools -> Stage MARC records for import.  Leaving all settings to default.  Output is:

MARC staging results :

    Processing bibliographic records
    10 records in file
    0 records not staged because of MARC error
    10 records staged
    Did not check for matches with existing records in catalog
    21 item records found and staged

(consistent with the content of batch)

Then I click on "Manage staged records" and, in the subsequent page, leave all settings to default and click on "Import this batch into the catalog".  Output is:

Completed import of records

Number of records added	10
Number of records updated	0
Number of records ignored	0
Number of items added	0
Number of items replaced	0
Number of items ignored because of duplicate barcode	0

Clearly the items have been ignored; confirmed by searching the OPAC: it finds the biblio records but no items.
Comment 5 Fridolin Somers 2019-03-07 08:57:30 UTC
(In reply to Fridolin SOMERS from comment #2)
> (In reply to Giuseppe Ciaccio from comment #0)
> > So it seems that the UNIMARC item field is correctly fetched when importing
> > the backup in the stage area, but discarded when the staged records are
> > pushed in the catalog.
> 
> I think I found why.
> The push to catalog is not allowed because the items have the same barcode
> which must be unique in catalog.

Ok it is not that.
Does instances A and B have the same branches ?
for items homebranch and holdingbranch
Comment 6 Giuseppe Ciaccio 2019-03-07 17:24:37 UTC
(In reply to Fridolin SOMERS from comment #5)
> (In reply to Fridolin SOMERS from comment #2)
> > (In reply to Giuseppe Ciaccio from comment #0)
> > > So it seems that the UNIMARC item field is correctly fetched when importing
> > > the backup in the stage area, but discarded when the staged records are
> > > pushed in the catalog.
> > 
> > I think I found why.
> > The push to catalog is not allowed because the items have the same barcode
> > which must be unique in catalog.
> 
> Ok it is not that.
> Does instances A and B have the same branches ?
> for items homebranch and holdingbranch

Bingo!  The two instances A and B have different branchcode and it seems that instance A refuses to import items whose branchcode does not match.  Verified by manually editing subfields 995$b and 995$c -- replacing some of the original branchcodes with the one of instance A, then A will import only those items with matching branchcodes.
Comment 7 Jonathan Druart 2019-05-04 14:10:46 UTC
I guess there is the error in a lost output. We would be able to handle that better on top of bug 22417.