Bug 27245 - bulkmarcimport.pl error 'Already in a transaction'
Summary: bulkmarcimport.pl error 'Already in a transaction'
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: master
Hardware: All All
: P5 - low major (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Keywords:
Depends on: 26518
Blocks:
  Show dependency treegraph
 
Reported: 2020-12-16 09:17 UTC by Fridolin Somers
Modified: 2021-06-14 21:34 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Text to go in the release notes:
Version(s) released in:
21.05.00,20.11.02,20.05.08,19.11.14


Attachments
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl (2.00 KB, patch)
2020-12-18 13:43 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl (2.06 KB, patch)
2020-12-30 15:31 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl (2.32 KB, patch)
2021-01-08 13:29 UTC, Nick Clemens
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Fridolin Somers 2020-12-16 09:17:24 UTC
When using script to import records misc/migration_tools/bulkmarcimport.pl, 
I get the error :
.DBD::mysql::db begin_work failed: Already in a transaction at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1588, <GEN45> line 1.

I thinks it is because of transaction in the script :
$dbh->{AutoCommit} = 0;

And C4::Biblio::AddBiblio launches a transation :
        $schema->txn_do(sub {

Records are imported well but its a scary error :/
Comment 1 Fridolin Somers 2020-12-16 09:27:14 UTC
Looks like it is since Bug 26518
Comment 2 Jonathan Druart 2020-12-18 13:43:46 UTC
Created attachment 114521 [details] [review]
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl

To prevent
DBD::mysql::db begin_work failed: Already in a transaction at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1588, <GEN45> line 1.

Test plan:
Export more than 100 records
Use bulkmarcimport (with -commit=10) to import them
Modify a record to make the import fail (for instance having a too long
lccn)
Use bulkmarcimport (with -commit=10) to import them
Notice that the import stops but that the imported record are imported
(a part from the last batch of 10)
Comment 3 Victor Grousset/tuxayo 2020-12-21 16:00:14 UTC
Without the patch I'm getting «Already in a transaction» even with a (supposedly) valid MARCXML file with 120 records. I just exported the first 120 records of the sample data in XML.


With the patch, with valid or invalid data, it's just importing one record:
kohadev-koha@kohadevbox:/kohadevbox/koha$ misc/migration_tools/bulkmarcimport.pl -commit 10 -file 120-records.xml 
.
1 MARC records done in 0.0640840530395508 seconds
kohadev-koha@kohadevbox:/kohadevbox/koha$ misc/migration_tools/bulkmarcimport.pl -commit 10 -file 120-records-one-invalid.xml
.
1 MARC records done in 0.0556449890136719 seconds


Maybe I'm missing a param from bulkmarcimport.pl? (-b doesn't help)
Comment 4 Fridolin Somers 2020-12-30 15:30:22 UTC
For me its ok, works as expected
Comment 5 Fridolin Somers 2020-12-30 15:31:13 UTC
Created attachment 114750 [details] [review]
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl

To prevent
DBD::mysql::db begin_work failed: Already in a transaction at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1588, <GEN45> line 1.

Test plan:
Export more than 100 records
Use bulkmarcimport (with -commit=10) to import them
Modify a record to make the import fail (for instance having a too long
lccn)
Use bulkmarcimport (with -commit=10) to import them
Notice that the import stops but that the imported record are imported
(a part from the last batch of 10)

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 6 Victor Grousset/tuxayo 2021-01-07 23:20:51 UTC
@Fridolin how did you use bulkmarcimport.pl ?
And how did you get the records? So I can test with then to check if my records were wrong.
Comment 7 Nick Clemens 2021-01-08 13:29:30 UTC
Created attachment 114958 [details] [review]
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl

To prevent
DBD::mysql::db begin_work failed: Already in a transaction at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1588, <GEN45> line 1.

Test plan:
1 - Export more than 100 records
2 - Use bulkmarcimport (with -commit=10) to import them
    perl misc/migration_tools/bulkmarcimport.pl -b -file testbmi.xml -v -m=MARCXML --commit=10
3 - Modify a record to make the import fail (for instance having a too long
lccn)
4 - Use bulkmarcimport (with -commit=10) to import them
    perl misc/migration_tools/bulkmarcimport.pl -b -file testbmi.xml -v -m=MARCXML --commit=10
5 - Notice that the import stops but that the imported record are imported (apart from the last batch of 10)

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 8 Jonathan Druart 2021-01-08 14:23:21 UTC
Pushed to master for 21.05, thanks to everybody involved!
Comment 9 Fridolin Somers 2021-01-11 11:05:31 UTC
Pushed to 20.11.x for 20.11.02
Comment 10 Andrew Fuerste-Henry 2021-01-14 18:58:58 UTC
Pushed to 20.05.x for 20.05.08
Comment 11 Victor Grousset/tuxayo 2021-01-22 01:01:10 UTC
Backported: Pushed to 19.11.x branch for 19.11.14