Bug 28152 - Hidden error when importing an item with an existing itemnumber
Summary: Hidden error when importing an item with an existing itemnumber
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Tools (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Jonathan Druart
QA Contact: Martin Renvoize
URL:
Keywords:
: 28128 (view as bug list)
Depends on:
Blocks: 17258 30831
  Show dependency treegraph
 
Reported: 2021-04-15 10:12 UTC by Jonathan Druart
Modified: 2023-06-08 22:26 UTC (History)
9 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
22.11.00, 22.05.01, 21.11.07


Attachments
Bug 28152: Fix import_items row creation if duplicate barcode (1.26 KB, patch)
2021-04-15 10:13 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 28152: Log the "duplicate item barcode" error (2.74 KB, patch)
2022-01-10 10:58 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 28152: Log the "duplicate item barcode" error (3.21 KB, patch)
2022-05-17 12:37 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 28152: Fix import_items row creation if duplicate barcode (1.40 KB, patch)
2022-05-21 16:21 UTC, Joonas Kylmälä
Details | Diff | Splinter Review
Bug 28152: Log the "duplicate item barcode" error (3.36 KB, patch)
2022-05-21 16:21 UTC, Joonas Kylmälä
Details | Diff | Splinter Review
Bug 28152: Fix import_items row creation if duplicate barcode (1.46 KB, patch)
2022-05-23 14:02 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 28152: Log the "duplicate item barcode" error (3.41 KB, patch)
2022-05-23 14:02 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 28152: Fix import_items row creation if duplicate barcode (1.52 KB, patch)
2022-05-24 06:48 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 28152: Log the "duplicate item barcode" error (3.47 KB, patch)
2022-05-24 06:48 UTC, Martin Renvoize
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2021-04-15 10:12:06 UTC
We are trying to insert "duplicate item barcode" into import_items.itemnumber (integer), it fails with "Incorrect integer value: 'duplicate item barcode' for column 'itemnumber' at row 1"

To reproduce:
Export a biblio with an item
Import it
=> The item is not added, and there is no new row in import_items.
Comment 1 Jonathan Druart 2021-04-15 10:13:34 UTC
Created attachment 119609 [details] [review]
Bug 28152: Fix import_items row creation if duplicate barcode

We are trying to insert "duplicate item barcode" into import_items.itemnumber (integer),
it fails with "Incorrect integer value: 'duplicate item barcode' for column 'itemnumber' at row 1"

To reproduce:
Export a biblio with an item
Import it
=> The item is not added, and there is no new row in import_items.

The error only appears in the log if you comment the close STDERR and
close STDOUT lines
Comment 2 Hayley Pelham 2021-12-21 22:30:49 UTC
Hi Jonathan,

I tried to test this but the patch didn't seem to fix the behaviour.

What I did:

1) Exported record with an item, confirmed contained 952 data
2) Staged for import with default parameters
3) Imported record with Always add items
4) import_items contained a new row

 import_items_id | import_record_id | itemnumber | branchcode | status | marcxml                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | import_error |
+-----------------+------------------+------------+------------+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+
|               1 |                1 |          0 | NULL       | error  | <?xml version="1.0" encoding="UTF-8"?>
<collection
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd"
  xmlns="http://www.loc.gov/MARC21/slim">

<record>
  <leader>00000    a              </leader>
  <datafield tag="952" ind1=" " ind2=" ">
    <subfield code="0">0</subfield>
    <subfield code="1">0</subfield>
    <subfield code="4">0</subfield>
    <subfield code="6">_</subfield>
    <subfield code="7">0</subfield>
    <subfield code="8">REF</subfield>
    <subfield code="9">192</subfield>
    <subfield code="a">FPL</subfield>
    <subfield code="b">FPL</subfield>
    <subfield code="c">GEN</subfield>
    <subfield code="d">2014-09-04</subfield>
    <subfield code="p">39999000003697</subfield>
    <subfield code="r">2014-09-04</subfield>
    <subfield code="w">2014-09-04</subfield>
    <subfield code="y">BK</subfield>
  </datafield>
</record>

</collection> | NULL         |

5) Checked imported record with View button in Manage staged MARC import and saw that there were no items associated with record
6) Applied patch and restarted services
7) Repeated steps 2-3
8) import_items contained a new row

|               2 |                2 |       NULL | NULL       | error  | <?xml version="1.0" encoding="UTF-8"?>
<collection
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd"
  xmlns="http://www.loc.gov/MARC21/slim">

<record>
  <leader>00000    a              </leader>
  <datafield tag="952" ind1=" " ind2=" ">
    <subfield code="0">0</subfield>
    <subfield code="1">0</subfield>
    <subfield code="4">0</subfield>
    <subfield code="6">_</subfield>
    <subfield code="7">0</subfield>
    <subfield code="8">REF</subfield>
    <subfield code="9">192</subfield>
    <subfield code="a">FPL</subfield>
    <subfield code="b">FPL</subfield>
    <subfield code="c">GEN</subfield>
    <subfield code="d">2014-09-04</subfield>
    <subfield code="p">39999000003697</subfield>
    <subfield code="r">2014-09-04</subfield>
    <subfield code="w">2014-09-04</subfield>
    <subfield code="y">BK</subfield>
  </datafield>
</record>

</collection> | NULL         |

9) Record still didn't have an item associated

Please let me know if I've missed some step!
Comment 3 Jonathan Druart 2022-01-03 11:09:16 UTC
Hi Hayley, what you are describing is the correct behaviour, the row in import_items should be there. The problem here is that, depending on the DBMS version, there is an error creating the row and it's missing.

I should have added more information as I don't remember exactly the context, but a guess is that it may be an error with a more recent version, like MySQL 8.
Comment 4 Katrin Fischer 2022-01-08 22:40:06 UTC
I saw this and I think I have been looking up that error in SQL at some point. I wonder if it got broken maybe by a change of sequence in parameters?

Instead of removing it, I think the "duplicate item barcode" should go into import_items.import_error that is NULL in Hayley's test.
Comment 5 Jonathan Druart 2022-01-10 10:16:31 UTC
(In reply to Katrin Fischer from comment #4)
> I saw this and I think I have been looking up that error in SQL at some
> point. I wonder if it got broken maybe by a change of sequence in parameters?
> 
> Instead of removing it, I think the "duplicate item barcode" should go into
> import_items.import_error that is NULL in Hayley's test.

Hum, might be. But this is since

commit 1dba9c6409d78cb1f90de6c1300cb5b63fb1b851
Date:   Wed Oct 10 14:21:22 2012 -0500
    Bug 7131: teach MARC import how to overlay items
Comment 6 Jonathan Druart 2022-01-10 10:17:36 UTC
Looking at the code this 'import_error' column is empty for the last 9 years. Not sure it makes much sense to have this single error now.
Comment 7 Katrin Fischer 2022-01-10 10:35:20 UTC
(In reply to Jonathan Druart from comment #6)
> Looking at the code this 'import_error' column is empty for the last 9
> years. Not sure it makes much sense to have this single error now.

Well, I deal a lot with old versions, I guess (and been around too long). Duplicate barcodes might appear without overlaying as well, just importing new data.

Would it be hard to move it to the error message?
Comment 8 Jonathan Druart 2022-01-10 10:58:32 UTC
Created attachment 129243 [details] [review]
Bug 28152: Log the "duplicate item barcode" error

Looking at the code this 'import_error' column is empty for the last 9 years. Not sure it makes much sense to have this single error now.

commit 1dba9c6409d78cb1f90de6c1300cb5b63fb1b851
Date:   Wed Oct 10 14:21:22 2012 -0500
    Bug 7131: teach MARC import how to overlay items
Comment 9 Jonathan Druart 2022-01-10 10:58:48 UTC
I failed to provide tests for this change.
Comment 10 David Cook 2022-05-12 03:46:48 UTC
(In reply to Jonathan Druart from comment #1)
> The error only appears in the log if you comment the close STDERR and
> close STDOUT lines

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30738 should be able to help there...
Comment 11 Joonas Kylmälä 2022-05-16 18:35:54 UTC
You can test this quite easily by first staging the record on intranet and then on the command line by running:

> $ perl misc/commit_file.pl --batch-number 8
> ... importing MARC records -- please wait
> C4::ImportBatch::BatchCommitItems(): DBI Exception: DBD::mysql::st execute failed: Incorrect integer value: 'duplicate item barcode' for column `koha_kohadev`.`import_items`.`itemnumber` at row 1 at /kohadevbox/koha/C4/ImportBatch.pm line 717

Note that the commit_file.pl might give other unrelated errors to this bug if you happen to give some record that it doesn't like but just ignore those...
Comment 12 Joonas Kylmälä 2022-05-16 18:45:26 UTC
Comment on attachment 129243 [details] [review]
Bug 28152: Log the "duplicate item barcode" error

Review of attachment 129243 [details] [review]:
-----------------------------------------------------------------

Found a bug, moving to FQA

::: C4/ImportBatch.pm
@@ +798,3 @@
>              $updsth->execute();
>              $num_items_errored++;
>          } else {

The code under this else block is unmodified, thus broken.
Comment 13 Jonathan Druart 2022-05-17 12:37:46 UTC
Created attachment 135070 [details] [review]
Bug 28152: Log the "duplicate item barcode" error

Looking at the code this 'import_error' column is empty for the last 9 years. Not sure it makes much sense to have this single error now.

commit 1dba9c6409d78cb1f90de6c1300cb5b63fb1b851
Date:   Wed Oct 10 14:21:22 2012 -0500
    Bug 7131: teach MARC import how to overlay items
Comment 14 Joonas Kylmälä 2022-05-21 16:21:47 UTC
Created attachment 135253 [details] [review]
Bug 28152: Fix import_items row creation if duplicate barcode

We are trying to insert "duplicate item barcode" into import_items.itemnumber (integer),
it fails with "Incorrect integer value: 'duplicate item barcode' for column 'itemnumber' at row 1"

To reproduce:
Export a biblio with an item
Import it
=> The item is not added, and there is no new row in import_items.

The error only appears in the log if you comment the close STDERR and
close STDOUT lines

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Comment 15 Joonas Kylmälä 2022-05-21 16:21:51 UTC
Created attachment 135254 [details] [review]
Bug 28152: Log the "duplicate item barcode" error

Looking at the code this 'import_error' column is empty for the last 9 years. Not sure it makes much sense to have this single error now.

commit 1dba9c6409d78cb1f90de6c1300cb5b63fb1b851
Date:   Wed Oct 10 14:21:22 2012 -0500
    Bug 7131: teach MARC import how to overlay items

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Comment 16 Nick Clemens 2022-05-23 14:02:28 UTC
Created attachment 135267 [details] [review]
Bug 28152: Fix import_items row creation if duplicate barcode

We are trying to insert "duplicate item barcode" into import_items.itemnumber (integer),
it fails with "Incorrect integer value: 'duplicate item barcode' for column 'itemnumber' at row 1"

To reproduce:
Export a biblio with an item
Import it
=> The item is not added, and there is no new row in import_items.

The error only appears in the log if you comment the close STDERR and
close STDOUT lines

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 17 Nick Clemens 2022-05-23 14:02:32 UTC
Created attachment 135268 [details] [review]
Bug 28152: Log the "duplicate item barcode" error

Looking at the code this 'import_error' column is empty for the last 9 years. Not sure it makes much sense to have this single error now.

commit 1dba9c6409d78cb1f90de6c1300cb5b63fb1b851
Date:   Wed Oct 10 14:21:22 2012 -0500
    Bug 7131: teach MARC import how to overlay items

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 18 Martin Renvoize 2022-05-24 06:48:44 UTC
Created attachment 135294 [details] [review]
Bug 28152: Fix import_items row creation if duplicate barcode

We are trying to insert "duplicate item barcode" into import_items.itemnumber (integer),
it fails with "Incorrect integer value: 'duplicate item barcode' for column 'itemnumber' at row 1"

To reproduce:
Export a biblio with an item
Import it
=> The item is not added, and there is no new row in import_items.

The error only appears in the log if you comment the close STDERR and
close STDOUT lines

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 19 Martin Renvoize 2022-05-24 06:48:48 UTC
Created attachment 135295 [details] [review]
Bug 28152: Log the "duplicate item barcode" error

Looking at the code this 'import_error' column is empty for the last 9 years. Not sure it makes much sense to have this single error now.

commit 1dba9c6409d78cb1f90de6c1300cb5b63fb1b851
Date:   Wed Oct 10 14:21:22 2012 -0500
    Bug 7131: teach MARC import how to overlay items

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 20 Martin Renvoize 2022-05-24 06:49:37 UTC
Adding an additional QA stamp as I'm using this to test the Unit test patch in bug 30831
Comment 21 Tomás Cohen Arazi 2022-05-31 12:00:46 UTC
Pushed to master for 22.11, thanks!
Comment 22 Lucas Gass 2022-06-06 14:22:52 UTC
Pushed to 22.05.x for 22.05.01
Comment 23 Arthur Suzuki 2022-06-07 10:10:42 UTC
pushed to 21.11.x for 21.11.07, thanks!
Comment 24 Victor Grousset/tuxayo 2022-06-25 21:44:02 UTC
Not backported to oldoldstable (21.05.x). Feel free to ask if it's needed.
Comment 25 David Cook 2022-06-27 23:08:58 UTC
*** Bug 28128 has been marked as a duplicate of this bug. ***