If a MARC file is staged and has found matching records already in Koha, but one of those records that was matched on is deleted before the staged records are imported, the import will stop cold at the missing record and just hang there indefinitely.
Created attachment 14985 [details] [review] Bug 9523 - Trying to import staged marc records where a matched bib has been deleted caused the import to hang Test Plan: 1) Stage a MARC records file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed.
Your patch changes the query to SELECT candidate_match_id LEFT JOIN biblio ON ( candidate_match_id = biblionumber ) FROM import_record_matches WHERE import_record_id = ? ORDER BY score DESC, candidate_match_id DESC; This is invalid SQL ... JOIN clauses come after FROM clauses. Even were the SQL statement constructed correctly, the left join by itself would make no difference to the query results, as the candidate_match_ids would get returned whether or not the matching bib was still attached. The patch might *appear* to work, but would have the effect of unconditionally all bibs in the import batch, regardless of matches. As a note for testing, I suggest using the command-line staging import tools, as they reproduce the problem *and* give useful output in the case of a failure. For example, if you follow the reproduction steps but use the command-line tools to commit the batch, you get the following error: $ misc/commit_file.pl --batch-number 5 ... importing MARC records -- please wait Empty String at /usr/lib/perl5/XML/LibXML/SAX/Parser.pm line 42
You are correct. I'll post up a corrected patch when I can. Kyle (In reply to comment #2) > Your patch changes the query to > > SELECT candidate_match_id > LEFT JOIN biblio ON ( candidate_match_id = biblionumber ) > FROM import_record_matches > WHERE import_record_id = ? > ORDER BY score DESC, candidate_match_id DESC; > > This is invalid SQL ... JOIN clauses come after FROM clauses. Even were the > SQL statement constructed correctly, the left join by itself would make no > difference to the query results, as the candidate_match_ids would get > returned whether or not the matching bib was still attached. > > The patch might *appear* to work, but would have the effect of > unconditionally all bibs in the import batch, regardless of matches. > > As a note for testing, I suggest using the command-line staging import > tools, as they reproduce the problem *and* give useful output in the case of > a failure. For example, if you follow the reproduction steps but use the > command-line tools to commit the batch, you get the following error: > > $ misc/commit_file.pl --batch-number 5 > ... importing MARC records -- please wait > Empty String at /usr/lib/perl5/XML/LibXML/SAX/Parser.pm line 42
Created attachment 15143 [details] [review] Bug 9523 - Trying to import staged marc records where a matched bib has been deleted caused the import to hang Test Plan: 1) Stage a MARC records file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed.
Created attachment 15144 [details] [review] Bug 9523 - Trying to import staged marc records where a matched bib has been deleted caused the import to hang Test Plan: 1) Stage a MARC records file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed.
Created attachment 16079 [details] [review] Bug 9523: importing staged bib records hangs if a matched bib has been deleted Test Plan: 1) Stage a MARC record file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed. Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Kyle, I cannot reproduce the issue. My test plan: 1/ Choose a biblio (biblionumber=XXX) 2/ Save it as MARC unicode/UTF-8 3/ Stage my record file with a matching rule (ISBN) and add the incoming record if matching. Screen is: 1 records in file 0 records not staged because of MARC error 1 records staged 1 records with at least one match in catalog per matching rule "ISBN" 1 item records found and staged 4/ Delete items and biblio XXX 5/ Import into the catalog Screen is: Number of records added 1 Number of records updated 0 Number of records ignored 0 Number of items added 1 Number of items ignored because of duplicate barcode 0 A new record with biblionumber=YYY is created. What I missed?
Odd, I can still replicate this bug on master: http://screencast.com/t/M5qWorUpl Kyle (In reply to comment #7) > Kyle, > I cannot reproduce the issue. > > My test plan: > 1/ Choose a biblio (biblionumber=XXX) > 2/ Save it as MARC unicode/UTF-8 > 3/ Stage my record file with a matching rule (ISBN) and add the incoming > record if matching. > Screen is: > 1 records in file > 0 records not staged because of MARC error > 1 records staged > 1 records with at least one match in catalog per matching rule "ISBN" > 1 item records found and staged > > 4/ Delete items and biblio XXX > 5/ Import into the catalog > Screen is: > Number of records added 1 > Number of records updated 0 > Number of records ignored 0 > Number of items added 1 > Number of items ignored because of duplicate barcode 0 > > A new record with biblionumber=YYY is created. > > What I missed?
(In reply to comment #8) > Odd, I can still replicate this bug on master: > http://screencast.com/t/M5qWorUpl > > Kyle Ok thanks. I can reproduce if I follow your way. I did not import twice the same record, but just one time an existing record. I thought it was the same thing. Marked as Passed QA.
Created attachment 17344 [details] [review] Bug 9523: importing staged bib records hangs if a matched bib has been deleted Test Plan: 1) Stage a MARC record file that will have matches with existing records 2) Delete the bib from Koha that was matched on 3) Attempt to import the records into Koha, the import will hang 4) Apply the patch 5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed. Signed-off-by: Galen Charlton <gmc@esilibrary.com> Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
(In reply to comment #9) > (In reply to comment #8) > > Odd, I can still replicate this bug on master: > > http://screencast.com/t/M5qWorUpl > > > > Kyle > > Ok thanks. I can reproduce if I follow your way. > I did not import twice the same record, but just one time an existing record. > I thought it was the same thing. > > Marked as Passed QA. I should have worded it more explicitly, my bad! Kyle
This patch has been pushed to master and 3.12.x.
Pushed to 3.10.x will be in 3.10.6