Until now, when you add order for a single record from a staged batch, duplicates are checked and if found you can choose 3 different treatments : adding a new order from existing record, adding a new record nevertheless or doing nothing. But when importing the batch all in once (Import all), there's no duplicate checking. This patch aims to solve this. When adding a batch of records to a basket, duplicates are skipped and an alert is displayed with a link to them so as they could be treated individually. In the case that all batch records match an existing record, you stay on the staged batch list, a warning is also visible to tell you that nothing had been imported. The "Import all" block is not any more displayed Test plan : You need the 2 test attached files TestFile1.mrc and TestFile2.elc (TestFile1 includes only the title "Amilec ou La graine d'hommes" that is also included in TestFile2) 1) go to “Stage MARC records for import” page, upload TestFile1 and stage it (select iso 5426 encoding). 2) Manage staged record and import the batch. 3) Make sure that the new record is indexed (depending to your indexing system and test platform). 4) Go back to go to “Stage MARC records for import” page upload TestFile2 and stage it (select iso 5426 encoding). 5) Go to acquisitions module and create a new basket. 6) From your basket, in the “Add order to basket block” choose 'From a staged file'. 7) Then click File2 (‘addorder button'). 8) Go down the "Import all" block and save. 9) You are redirected to the basket page : a warning is displayed to tell you that some duplicates have been found and skipped. There's a link on the warning throughout you can go back to the list of remaining records and treat them individually if necesary. 10) Click the link : you fall upon the title of TestFile1 (of course as it's a duplicate). 11) Check that the imported records have been indexed. 11) Go down the "Import all" block and save. 12) A warning is displayed saying that no records have been imported because they all match an existing record. The “Import all” block is not any more visible. Olivier Crouzet
Created attachment 27031 [details] [review] Bug 12074 Filter duplicates when adding a batch from a staged file
Created attachment 27032 [details] test file 1
Created attachment 27033 [details] test file 2
*** Bug 6386 has been marked as a duplicate of this bug. ***
Bug 7180 has been (finally) pushed. This one does not apply anymore.
Hi there! Our librarians are having hard time because this new feature doesn't check for duplicates when making acquisitions from this staged file. They share your pain, in addition to that we are providing a few more improvements to ease acquisition decisions from the staging file. 1. Provide visual cues of which lineitems are already imported and show a "map" of existing Items in the consortium (of each branch) non-disruptively. 2. Display the default price fetched from the MARC record in the staged file listing. 3. (Having found this bug) Add a checkbox for each matched record whether or not to overwrite the existing record with this file or not. Defaults to not overwriting. As a possible improvement... we have an in-house patch which makes MARC transformations based on Matcher rules so you can automatically transform the imported MARC based on Matcher rules, for ex move the agerestriction from incoming Records 049c => 521a in Koha. Different vendors provide some information in different fields. If you want to test it I can rebase it for you. It might be better to do the transformations during the stage file process though.
(In reply to Olli-Antti Kivilahti from comment #6) > Hi there! Goedemorgen > As a possible improvement... we have an in-house patch which makes MARC > transformations based on Matcher rules so you can automatically transform > the imported MARC based on Matcher rules, for ex move the agerestriction > from incoming Records 049c => 521a in Koha. Several people are also / have been working on improving the marc mod templates functionality..
(In reply to M. de Rooy from comment #7) > (In reply to Olli-Antti Kivilahti from comment #6) > > Hi there! > Goedemorgen > > > As a possible improvement... we have an in-house patch which makes MARC > > transformations based on Matcher rules so you can automatically transform > > the imported MARC based on Matcher rules, for ex move the agerestriction > > from incoming Records 049c => 521a in Koha. > > Several people are also / have been working on improving the marc mod > templates functionality.. Hi! I have heard of it as well, but the results are not available NOW() :) Also I think their work is a bit too complex for our needs. Shared our feature here: Bug 13181
Is this still valid?
We just need to find time to check ... I'll also take a look at Bug 13181
Regarding Bug 13181 you should focus on the community effort of using MARC Modification Templates. My hack is obsoleted and will be replaced by the community tool.
In fact , the bug 13182 is most interesting for us. We want to keep the possibility to see the duplicates and to choose to order it when adding a full batch in a order.
Created attachment 36144 [details] [review] Bug 12074 - Filter duplicates when adding a full batch from a staged file patch rebased
the patch is applied correctly. When adding a batch of documents to a basket , duplicates are ignored and correctly a visual alert is displayed with a link to them so they can be treated so individual . Signed off
Patch tested with a sandbox, by JA <aloi54@live.fr>
Created attachment 36647 [details] [review] Bug 12074 Filter duplicates when adding a batch from a staged file When adding a batch of records to a basket, duplicates are skipped and an alert is displayed with a link to them so as they could be treated individually. Signed-off-by: JA <aloi54@live.fr>
Created attachment 36948 [details] [review] Bug 12074 Filter duplicates when adding a batch from a staged file When adding a batch of records to a basket, duplicates are skipped and an alert is displayed with a link to them so as they could be treated individually. Test plan : You need the 2 test attached files TestFile1.mrc and TestFile2.elc (TestFile1 includes only the title "Amilec ou La graine d'hommes" that is also included in TestFile2) 1) go to “Stage MARC records for import” page, upload TestFile1 and stage it (select iso 5426 encoding). 2) Manage staged record and import the batch. 3) Make sure that the new record is indexed (depending to your indexing system and test platform). 4) Go back to go to “Stage MARC records for import” page upload TestFile2 and stage it (select iso 5426 encoding). 5) Go to acquisitions module and create a new basket. 6) From your basket, in the “Add order to basket block” choose 'From a staged file'. 7) Then click File2 (‘addorder button'). 8) Go down the "Import all" block and save. 9) You are redirected to the basket page : a warning is displayed to tell you that some duplicates have been found and skipped. There's a link on the warning throughout you can go back to the list of remaining records and treat them individually if necesary. 10) Click the link : you fall upon the title of TestFile1 (of course as it's a duplicate). 11) Check that the imported records have been indexed. 11) Go down the "Import all" block and save. 12) A warning is displayed saying that no records have been imported because they all match an existing record. The “Import all” block is not any more visible. Signed-off-by: JA <aloi54@live.fr> Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Patch pushed to master. Thanks Lyon3 Team!