Created attachment 191850 [details] [review] Bug 41681: bulkmarcimport.pl reports an incorrect number of MARC records processed When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file. To test: 1. Download the joined MARC file containing 5 records. 2. Run the import command: ./misc/migration_tools/bulkmarcimport.pl -b --file=marc_file.mrc 3. Check the final script report ==> 11 MARC records done in 0.566349983215332 seconds Expected output should show 5 records 4. Apply the patch 5. Repeat step 2, 3 ==> The correct number is shown Created attachment 193360 [details] [review] Bug 41681: bulkmarcimport.pl reports an incorrect number of MARC records processed When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file. To test: 1. Download the joined MARC file containing 5 records. 2. Run the import command: ./misc/migration_tools/bulkmarcimport.pl -b --file=marc_file.mrc 3. Check the final script report ==> 11 MARC records done in 0.566349983215332 seconds Expected output should show 5 records 4. Apply the patch 5. Repeat step 2, 3 ==> The correct number is shown Signed-off-by: Janusz Kaczmarek <januszop@gmail.com> *** Bug 41147 has been marked as a duplicate of this bug. *** Created attachment 194964 [details]
File with 2 MARC records where the first is not valid
From bug 41147 there is an important comment from Nick in his patch: # We increment here, if a record is skipped because we can't load form the batch or can't be converted # it still counts against the limit if a number to process is passed And so I created a file with the first record to be broken to test against param n=1 which would mean that only 1 record should be imported from the given file. As the first record is broken the script then should say 0 records imported (though 1 record processed) But the output is still like '1 MARC records done in ...' so I think there needs to be done a bit more or am I misinterpreting the number param to count only successful records? Tested via: perl misc/migration_tools/bulkmarcimport.pl -b -v -n=1 -m MARCXML --file broken.xml Created attachment 195081 [details] [review] Bug 41681: bulkmarcimport.pl reports an incorrect number of MARC records processed When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file. To test: 1. Download the joined MARC file containing 5 records. 2. Run the import command: ./misc/migration_tools/bulkmarcimport.pl -b --file=marc_file.mrc 3. Check the final script report ==> 11 MARC records done in 0.566349983215332 seconds Expected output should show 5 records 4. Apply the patch 5. Repeat step 2, 3 ==> The correct number is shown Signed-off-by: Janusz Kaczmarek <januszop@gmail.com> Created attachment 195082 [details] [review] Bug 41681: (follow-up) Fix record number count when broken records are processed When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file. This patch fixes 3 issues - broken (invalid) records from batch get now counted (record_number) - when using the number param (n) to limit the number of records to be processed valid and invalid records get counted - when using th offset param (o) the scipt errored when broken records were part of the offset. To test: 1. Download the MARC file with the broken record 2. Run the following import commands and compare the counted import numbers: a) all records perl misc/migration_tools/bulkmarcimport.pl -b -v -m MARCXML --file 2_records_1st_broken.xml 4 MARC records done in 0.0421221256256104 seconds b) limit to 1 record in total perl misc/migration_tools/bulkmarcimport.pl -b -v -n 1 -m MARCXML --file 2_records_1st_broken.xml 4 MARC records done in 0.0420489311218262 seconds c) limit to 1 record offset 1 record perl misc/migration_tools/bulkmarcimport.pl -b -v -n 1 -o 1 -m MARCXML --file 2_records_1st_broken.xml :2: parser error : Opening and ending tag mismatch: leader2 line 2 and leader <leader2>00216nam a22001097a 4500</leader> 3. Apply the patch 4. Repeat steps in 2 and compare the results a) 2 MARC records done in 0.0405838489532471 seconds b) 1 MARC records done in 0.00948596000671387 seconds * c) 1 MARC records done in 0.0294840335845947 seconds *though this one was skipped |
Created attachment 191849 [details] A marc file containing 5 records When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file. To reproduce: 1. Download the joined MARC file containing 5 records. 2. Run the import command: ./misc/migration_tools/bulkmarcimport.pl -b --file=marc_file.mrc 3. Check the final script report ==> 11 MARC records done in 0.566349983215332 seconds The reported number is incorrect