Bug 26521

Summary: Import_biblios table fields differ from biblio table fields
Product: Koha Reporter: Nick Clemens (kidclamp) <nick>
Component: ToolsAssignee: Bugs List <koha-bugs>
Status: NEW --- QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low    
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Description Nick Clemens (kidclamp) 2020-09-23 17:54:02 UTC
The fields in import_biblios are much shorter and stricter than those in the biblios.

The can cause odd behaviour where records import successfully but don't appear correctly in the import results

To recreate:
(assuming you are using sample date)
1 - Tools->export data
2 - Select nothing and click "Export bibliographic records"
3 - Save the file
4 - Tools->Stage MARC for import
5 - Stage the file - turn off matching and item processing
6 - Manage the batch
7 - Click Show -> All in the datatable
8 - Browse through and note some records show 'null' for title
9 - Import the batch
10 - Note the null titles don't show a biblionumber
11 - Search for one of them e.g. "Forest management div"
12 - It returns in results and is a valid Koha record, not the biblionumber
13 - Check the db:
     SELECT * FROM import_biblios WHERE matched_biblionumber=XXXX
14 - No import_biblio entry

Error during import:
[Wed Sep 23 17:53:20.821380 2020] [cgi:error] [pid 134046] [client 172.18.0.1:57228] AH01215: [Wed Sep 23 17:53:20 2020] stage-marc-import.pl: DBD::mysql::st execute failed: Data too long for column 'title' at row 1 [for Statement "INSERT INTO import_biblios (import_record_id, title, author, isbn, issn) VALUES (?, ?, ?, ?, ?)" with ParamValues: 0='474', 1="Pictura mural\xc4\x83  din nordul Moldovei: modificari estetice si restarare = Mural painting in the north of Moldavia: aesthetic modification and restoration.", 2="Dina, Anca (ed.).", 3="973177212X", 4=undef] at /kohadevbox/koha/C4/ImportBatch.pm line 1640.: /kohadevbox/koha/tools/stage-marc-import.pl, referer: http://localhost:8081/cgi-bin/koha/tools/stage-marc-import.pl

MariaDB [koha_kohadev]> describe import_biblios;
+----------------------+--------------+------+-----+---------+-------+
| Field                | Type         | Null | Key | Default | Extra |
+----------------------+--------------+------+-----+---------+-------+
| import_record_id     | int(11)      | NO   | MUL | NULL    |       |
| matched_biblionumber | int(11)      | YES  | MUL | NULL    |       |
| control_number       | varchar(25)  | YES  |     | NULL    |       |
| original_source      | varchar(25)  | YES  |     | NULL    |       |
| title                | varchar(128) | YES  | MUL | NULL    |       |
| author               | varchar(80)  | YES  |     | NULL    |       |
| isbn                 | varchar(30)  | YES  | MUL | NULL    |       |
| issn                 | varchar(9)   | YES  |     | NULL    |       |
| has_items            | tinyint(1)   | NO   |     | 0       |       |
+----------------------+--------------+------+-----+---------+-------+

MariaDB [koha_kohadev]> describe biblio;
+---------------+-------------+------+-----+-------------------+-----------------------------+
| Field         | Type        | Null | Key | Default           | Extra                       |
+---------------+-------------+------+-----+-------------------+-----------------------------+
| biblionumber  | int(11)     | NO   | PRI | NULL              | auto_increment              |
| frameworkcode | varchar(4)  | NO   |     |                   |                             |
| author        | longtext    | YES  |     | NULL              |                             |
| title         | longtext    | YES  |     | NULL              |                             |
| medium        | longtext    | YES  |     | NULL              |                             |
| subtitle      | longtext    | YES  |     | NULL              |                             |
| part_number   | longtext    | YES  |     | NULL              |                             |
| part_name     | longtext    | YES  |     | NULL              |                             |
| unititle      | longtext    | YES  |     | NULL              |                             |
| notes         | longtext    | YES  |     | NULL              |                             |
| serial        | tinyint(1)  | YES  |     | NULL              |                             |
| seriestitle   | longtext    | YES  |     | NULL              |                             |
| copyrightdate | smallint(6) | YES  |     | NULL              |                             |
| timestamp     | timestamp   | NO   |     | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| datecreated   | date        | NO   |     | NULL              |                             |
| abstract      | longtext    | YES  |     | NULL              |                             |
+---------------+-------------+------+-----+-------------------+-----------------------------+
16 rows in set (0.00 sec)

MariaDB [koha_kohadev]> describe biblioitems;
+-----------------------+--------------+------+-----+-------------------+-----------------------------+
| Field                 | Type         | Null | Key | Default           | Extra                       |
+-----------------------+--------------+------+-----+-------------------+-----------------------------+
| biblioitemnumber      | int(11)      | NO   | PRI | NULL              | auto_increment              |
| biblionumber          | int(11)      | NO   | MUL | 0                 |                             |
| volume                | longtext     | YES  |     | NULL              |                             |
| number                | longtext     | YES  |     | NULL              |                             |
| itemtype              | varchar(10)  | YES  | MUL | NULL              |                             |
| isbn                  | longtext     | YES  | MUL | NULL              |                             |
| issn                  | longtext     | YES  | MUL | NULL              |                             |
| ean                   | longtext     | YES  | MUL | NULL              |                             |
| publicationyear       | mediumtext   | YES  |     | NULL              |                             |
| publishercode         | varchar(255) | YES  | MUL | NULL              |                             |
| volumedate            | date         | YES  |     | NULL              |                             |
| volumedesc            | mediumtext   | YES  |     | NULL              |                             |
| collectiontitle       | longtext     | YES  |     | NULL              |                             |
| collectionissn        | mediumtext   | YES  |     | NULL              |                             |
| collectionvolume      | longtext     | YES  |     | NULL              |                             |
| editionstatement      | mediumtext   | YES  |     | NULL              |                             |
| editionresponsibility | mediumtext   | YES  |     | NULL              |                             |
| timestamp             | timestamp    | NO   | MUL | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| illus                 | varchar(255) | YES  |     | NULL              |                             |
| pages                 | varchar(255) | YES  |     | NULL              |                             |
| notes                 | longtext     | YES  |     | NULL              |                             |
| size                  | varchar(255) | YES  |     | NULL              |                             |
| place                 | varchar(255) | YES  |     | NULL              |                             |
| lccn                  | varchar(25)  | YES  |     | NULL              |                             |
| url                   | mediumtext   | YES  |     | NULL              |                             |
| cn_source             | varchar(10)  | YES  |     | NULL              |                             |
| cn_class              | varchar(30)  | YES  |     | NULL              |                             |
| cn_item               | varchar(10)  | YES  |     | NULL              |                             |
| cn_suffix             | varchar(10)  | YES  |     | NULL              |                             |
| cn_sort               | varchar(255) | YES  |     | NULL              |                             |
| agerestriction        | varchar(255) | YES  |     | NULL              |                             |
| totalissues           | int(10)      | YES  |     | NULL              |                             |
+-----------------------+--------------+------+-----+-------------------+-----------------------------+
32 rows in set (0.01 sec)