Created attachment 2724 [details] version with new functionality commit_biblios_file.pl needs a way to revert a batch in bulk. I have done this code and will attache it to bug for review and potential inclusion. The basis for this enhancement was the a bad batch of MARC records that had some unknown Unicode characters in them which cause the import function from the web interface to fail after importing a portion of the records. At that point you are completely stuck. You can't complete the import due to the errors and there is no way to *revert* either. You are stuck with a half imported batch. My change is to add a --revert flag to commit_biblios_file.pl which provides an interface to the pre-existing BatchRevertBibRecords function. I have also added a --progress-interval flag to allow the default commit interval to be changed to aid in debugging bad import batches.
Created attachment 2725 [details] [review] Proposed patch
Pushed to new/enh/bug_5332 along with Galens follow up patches, please test and report so I can merge to master
(In reply to comment #2) > Pushed to new/enh/bug_5332 along with Galens follow up patches, please test and > report so I can merge to master I'm new to this process. Do you need me to test this? If so, I'm not sure how to work with the referenc new/enh/bug_5332. Pointers welcome.
Doug, are you running a development server or a test server anywhere? If so, and if it is running from git, you can do this git checkout -b bug_5332 origin/new/enh/bug_5332 And you will now be running that branch. If you aren't thats fine, just let us know and we will get someone else to test.
(In reply to comment #4) > Doug, are you running a development server or a test server anywhere? > > If so, and if it is running from git, you can do this > > git checkout -b bug_5332 origin/new/enh/bug_5332 > > And you will now be running that branch. If you aren't thats fine, just let us > know and we will get someone else to test. Sorry, at the moment I am running a stock 3.2. We don't have a QA environment yet. I'll try to get more organized and git-aware as I make more changes, but for now I think it will be faster for someone else. Do you need a duff marc import file (with some broken unicode in it) to create a broken import?
Yes please a attach a file that breaks it currently, and we can use that to test that it is now fixed.
Created attachment 2747 [details] Wrong file - please ignore Gzip'd MARC21 import file. Use this to trigger a partially imported batch that will not import and cannot be reverted (because its not been fully imported - This is probably a bug in and of itself!). There are 1443 MARC records in this file. I believe 28 are bad. The first bad entry in about record 9.
Created attachment 2748 [details] Gzip of MARC import file with bad records This gzip'd MARC21 import file has some bad unicode characters in it that will cause the import to abort part way through. At that point the batch import is stuck. It can't complete due to the errors and cannot be reverted from the web UI since it has not completed. (take8.marc.gz replaces take7.marc.gz which was erroneously uploaded yesterday.)
Now I am not sure which one was the problem import file (take7 - now marked obsolete, or take 8). But its one of them. I will try to get a test system up here so I can make a definitive statement.
Still awaiting testing
Created attachment 10232 [details] [review] Add batch revert capability (--revert) to commit_biblios_file.pl. Also add --progress-interval to facilitate debugging bad MARC batches. The basis for this enhancement was a bad batch of MARC records that had some unknown Unicode characters in them which caused the batch import function from the web interface to abort after importing only a portion of the records. At this point you are completely stuck using the web user interface. You can't go forward because the only option is to try the import again from the beginning (which fails again). You cannot revert either since there is no way to revert a partially committed batch. This change uses the existing BatchRevertBibRecords function. Bug reference: 5332 Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Rebased to current master 2012-06-11 - please test.
This functionality is also added by bug 2060. I overlooked this patch, and wrote more-or-less identical code when I needed the ability to revert from the command line to test authority import.
The main functionality of this patch (the --revert option) is covered by the patch for Bug 2060 (which is already signed off), as pointed out by Jared in Comment 12. I'm marking this bug as a duplicate of that bug. The present patch does introduce another option (--progress-interval) which is not present in the patches for Bug 2060, so that might be implemented on top of 2060 if someone wants to. *** This bug has been marked as a duplicate of bug 2060 ***