Bug 27245

Summary: bulkmarcimport.pl error 'Already in a transaction'
Product: Koha Reporter: Fridolin Somers <fridolin.somers>
Component: Command-line UtilitiesAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: major    
Priority: P5 - low CC: andrewfh, caroline.cyr-la-rose, jonathan.druart, kyle, robin, victor
Version: master   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29325
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
21.05.00,20.11.02,20.05.08,19.11.14
Bug Depends on: 26518    
Bug Blocks:    
Attachments: Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl

Description Fridolin Somers 2020-12-16 09:17:24 UTC
When using script to import records misc/migration_tools/bulkmarcimport.pl, 
I get the error :
.DBD::mysql::db begin_work failed: Already in a transaction at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1588, <GEN45> line 1.

I thinks it is because of transaction in the script :
$dbh->{AutoCommit} = 0;

And C4::Biblio::AddBiblio launches a transation :
        $schema->txn_do(sub {

Records are imported well but its a scary error :/
Comment 1 Fridolin Somers 2020-12-16 09:27:14 UTC
Looks like it is since Bug 26518
Comment 2 Jonathan Druart 2020-12-18 13:43:46 UTC
Created attachment 114521 [details] [review]
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl

To prevent
DBD::mysql::db begin_work failed: Already in a transaction at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1588, <GEN45> line 1.

Test plan:
Export more than 100 records
Use bulkmarcimport (with -commit=10) to import them
Modify a record to make the import fail (for instance having a too long
lccn)
Use bulkmarcimport (with -commit=10) to import them
Notice that the import stops but that the imported record are imported
(a part from the last batch of 10)
Comment 3 Victor Grousset/tuxayo 2020-12-21 16:00:14 UTC
Without the patch I'm getting «Already in a transaction» even with a (supposedly) valid MARCXML file with 120 records. I just exported the first 120 records of the sample data in XML.


With the patch, with valid or invalid data, it's just importing one record:
kohadev-koha@kohadevbox:/kohadevbox/koha$ misc/migration_tools/bulkmarcimport.pl -commit 10 -file 120-records.xml 
.
1 MARC records done in 0.0640840530395508 seconds
kohadev-koha@kohadevbox:/kohadevbox/koha$ misc/migration_tools/bulkmarcimport.pl -commit 10 -file 120-records-one-invalid.xml
.
1 MARC records done in 0.0556449890136719 seconds


Maybe I'm missing a param from bulkmarcimport.pl? (-b doesn't help)
Comment 4 Fridolin Somers 2020-12-30 15:30:22 UTC
For me its ok, works as expected
Comment 5 Fridolin Somers 2020-12-30 15:31:13 UTC
Created attachment 114750 [details] [review]
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl

To prevent
DBD::mysql::db begin_work failed: Already in a transaction at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1588, <GEN45> line 1.

Test plan:
Export more than 100 records
Use bulkmarcimport (with -commit=10) to import them
Modify a record to make the import fail (for instance having a too long
lccn)
Use bulkmarcimport (with -commit=10) to import them
Notice that the import stops but that the imported record are imported
(a part from the last batch of 10)

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 6 Victor Grousset/tuxayo 2021-01-07 23:20:51 UTC
@Fridolin how did you use bulkmarcimport.pl ?
And how did you get the records? So I can test with then to check if my records were wrong.
Comment 7 Nick Clemens 2021-01-08 13:29:30 UTC
Created attachment 114958 [details] [review]
Bug 27245: Replace AutoCommit=0 with txn_begin in bulkmarcimport.pl

To prevent
DBD::mysql::db begin_work failed: Already in a transaction at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1588, <GEN45> line 1.

Test plan:
1 - Export more than 100 records
2 - Use bulkmarcimport (with -commit=10) to import them
    perl misc/migration_tools/bulkmarcimport.pl -b -file testbmi.xml -v -m=MARCXML --commit=10
3 - Modify a record to make the import fail (for instance having a too long
lccn)
4 - Use bulkmarcimport (with -commit=10) to import them
    perl misc/migration_tools/bulkmarcimport.pl -b -file testbmi.xml -v -m=MARCXML --commit=10
5 - Notice that the import stops but that the imported record are imported (apart from the last batch of 10)

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 8 Jonathan Druart 2021-01-08 14:23:21 UTC
Pushed to master for 21.05, thanks to everybody involved!
Comment 9 Fridolin Somers 2021-01-11 11:05:31 UTC
Pushed to 20.11.x for 20.11.02
Comment 10 Andrew Fuerste-Henry 2021-01-14 18:58:58 UTC
Pushed to 20.05.x for 20.05.08
Comment 11 Victor Grousset/tuxayo 2021-01-22 01:01:10 UTC
Backported: Pushed to 19.11.x branch for 19.11.14