Bug 18482 - False duplicates detected on adding a batch from a stage file
Summary: False duplicates detected on adding a batch from a stage file
Status: Pushed to Stable
Alias: None
Product: Koha
Classification: Unclassified
Component: Acquisitions (show other bugs)
Version: master
Hardware: All All
: P5 - low major (vote)
Assignee: Koha Team Lyon 3
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
Depends on: 12074
Blocks:
  Show dependency treegraph
 
Reported: 2017-04-22 16:43 UTC by Koha Team Lyon 3
Modified: 2017-07-31 11:07 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
False duplicates detected on adding a batch from a stage file (1.79 KB, application/x-extension-mrc)
2017-04-22 16:43 UTC, Koha Team Lyon 3
Details
TestFile1 (1.79 KB, application/x-extension-mrc)
2017-04-22 16:44 UTC, Koha Team Lyon 3
Details
TestFile2 (13.68 KB, application/x-extension-elc)
2017-04-22 16:45 UTC, Koha Team Lyon 3
Details
Bug 18482 False duplicates detected on adding a batch from a stage file (1.88 KB, patch)
2017-04-22 16:50 UTC, Koha Team Lyon 3
Details | Diff | Splinter Review
Bug 18482 False duplicates detected on adding a batch from a stage file (1.93 KB, patch)
2017-04-23 23:23 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 18482 False duplicates detected on adding a batch from a stage file (2.00 KB, patch)
2017-04-24 17:30 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 18482 False duplicates detected on adding a batch [16.11.x] (2.09 KB, patch)
2017-05-19 13:08 UTC, Koha Team Lyon 3
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Koha Team Lyon 3 2017-04-22 16:43:20 UTC
Created attachment 62572 [details]
False duplicates detected on adding a batch from a stage file

When adding a batch from a stage file with defaut matching on
title/author, if a duplicate is detected, all following records
treated in the batch are discarded from import even if they are not duplicates.

Checking for duplicate now lies on duplinbatch value that is not reset at each loop (In the initial commit, discarding record was directly tight to FindDuplicate return). So from the point when duplinbatch is defined, all the following biblios are discarded as duplicate whatever FindDuplicate return is.
This patch adds a distinct flag.

Test plan :

You need the 2 test attached files TestFile1.mrc and TestFile2.elc (TestFile1 includes only the title "Amilec ou La graine d'hommes" that is also included in TestFile2). To save them, right click on bugzilla link and do 'Save target as'.

1) go to “Stage MARC records for import” page, upload TestFile1, select iso 5426 encoding and stage it for Import. 
2) Click again 'Stage MARC records' button and do the same for TestFile2.elc
3) Go to acquisitions module, enter a vendor name and create a new basket.
4) From your basket, click 'Add to basket', then in the “Add order to basket window/block” choose 'From a staged file'.
5) Then click ‘Add orders' button on File1.
6) Leave default matching on Title/Author, tick the title, add a location in second tag and fund in third tag and Save. 
7) Make sure that the new record is indexed (depending to your indexing
system and test platform).
8) Redo step 4 and step 5 this time on TestFile2.
9) Leave default matching on Title/Author, select all titles, add a location in second tag, a fund in third tag and Save. 
10) A warning is displayed saying that no records have been imported because they all match an existing record. The “Import all” block is not any more visible.
11) Apply the patch
12) Refresh 'Add orders from' page (File2) and redo step 9
13) You are redirected to the basket page : a warning is displayed to tell you that some duplicates have been found and skipped.
There's a link on the warning throughout you can go back to the list of remaining records and treat them individually if necesary.
14) Click the link : you fall upon the only title that is a very duplicate (record of TestFile1).
15) Be happy
16) Sign off

Olivier Crouzet
Comment 1 Koha Team Lyon 3 2017-04-22 16:44:50 UTC
Created attachment 62573 [details]
TestFile1
Comment 2 Koha Team Lyon 3 2017-04-22 16:45:42 UTC
Created attachment 62574 [details]
TestFile2
Comment 3 Koha Team Lyon 3 2017-04-22 16:50:04 UTC
Created attachment 62575 [details] [review]
Bug 18482 False duplicates detected on adding a batch from a stage file
Comment 4 Koha Team Lyon 3 2017-04-22 16:52:35 UTC
This patch must be added from version 3.22.13 and higher
Comment 5 Alex Buckley 2017-04-23 23:23:28 UTC
Created attachment 62581 [details] [review]
Bug 18482 False duplicates detected on adding a batch from a stage file

When adding a batch from a stage file with defaut matching on
title/author, if a duplicate is detected, all following records
treated in the batch are discarded from import even if they are not duplicates

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Comment 6 Jonathan Druart 2017-04-24 17:30:09 UTC
Created attachment 62621 [details] [review]
Bug 18482 False duplicates detected on adding a batch from a stage file

When adding a batch from a stage file with defaut matching on
title/author, if a duplicate is detected, all following records
treated in the batch are discarded from import even if they are not duplicates

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 7 Kyle M Hall 2017-04-28 10:51:19 UTC
Pushed to master for 17.05, thanks Olivier!
Comment 8 Katrin Fischer 2017-05-13 12:06:25 UTC
There is a conflict in acqui/addorderiso2709.pl when applying this to 16.11.x, can you please check?
Comment 9 Koha Team Lyon 3 2017-05-19 13:08:36 UTC
Created attachment 63579 [details] [review]
Bug 18482 False duplicates detected on adding a batch [16.11.x]

add specific patch for 16.11.x. Just remove the biblio_count line that seems to been added more lately
Comment 10 Katrin Fischer 2017-06-05 13:32:48 UTC
Thanks for the rebased patch!

This patch has been pushed to 16.11.x and will be in 16.11.09.
Comment 11 Mason James 2017-07-31 11:07:28 UTC
Pushed to 16.05.x, for 16.05.15 release