Bug 13182

Summary: When placing an order from a staged file, check for duplicates, display availability information, choose to overlay matched records.
Product: Koha Reporter: Olli-Antti Kivilahti <olli-antti.kivilahti>
Component: AcquisitionsAssignee: Olli-Antti Kivilahti <olli-antti.kivilahti>
Status: ASSIGNED --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: koha, mtompset
Version: Main   
Hardware: All   
OS: All   
See Also: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12074
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 13182 - When placing an order from a staged file, check for duplicates, display availability information, choose to overlay matched records.
pure awesomeness of this feature showcased.
Bug 13182 - When placing an order from a staged file, check for duplicates, display availability information, choose to overlay matched records.
Bug 13182 - When placing an order from a staged file, remove dead code
Bug 13182 - When placing an order from a staged file, check for duplicates, display availability information.
Bug 13182 - Overlay an existing Biblio with the acquired import_record
Bug 13182 - Overlay an existing Biblio with the acquired import_record
Bug 13182 - Remove really dead code.
Bug 13182 - When placing an order from a staged file, check for duplicates, display availability information.
Bug 13182 - Overlay an existing Biblio with the acquired import_record
Bug 13182 - Remove really dead code.

Description Olli-Antti Kivilahti 2014-10-31 15:53:09 UTC
When making acquisitions from one staged file, by many branches as a joint effort, it is important to be able to make orders from the same import_record.

This places some extra requirements.
1. An status = 'imported' import_record needs to be shown in the stage_file so other branches can order from it as well. So we can't "hide" a import_record from the staged_file after an order is placed.
2. Having imported one import_record, that import_record must know it is matched to the just imported Biblio, so next acquirers know that it is already in our catalog.
3. We need to know which other branches have already ordered the item so we wont end up ordering too many copies of one Biblio to the consortium. Thus we color each row and show the count of Items in the current user's branch and the total amount of Items in the whole consortium. Additional tooltip listing of Item counts by Branch is displayed so we know are there any such Items near us.
4. If we know we already have a matching biblio in Koha, we can optionally overlay the existing Biblio with the new one. (We don't really need it but looks like Bug 12074 needs it). Also a good hook to attach more advanced automatic overlaying/transformation mechanisms.

TEST PLAN:

SETUP:

0a. Define a Matcher from admin/matching-rules.pl. We think good starting values are:
rules, score
020a, 1000
024a, 1000
028a, 1000
245a, 500
240a, 500
110a, 300
100a, 300
Matching threshold 800

0b. Stage a MARC file.

1. Start making an order from a staged_file, stop when you see the
   import_records after you selected the batch to order from at
   acqui/addorderiso2709.pl?import_batch_id=50...
2. If you have matches, they are colored and the Item counts are visible.
   If not, order any item and return to the previous view and reload.
3. Observe that you should now have atleast one match. The match displays the
   amount of items in your branch, in your consortium, and a tooltip from the
   record title, showing a breakdown of items by branch.
4. When you add more orders, you get more order lines, but all orders are for
   the same Biblio, thus you don't get many Biblios, one per order.

TODO: Unit tests.
TODO: Better test plan.
Comment 1 Olli-Antti Kivilahti 2014-10-31 15:56:02 UTC Comment hidden (obsolete)
Comment 2 Olli-Antti Kivilahti 2014-10-31 16:02:26 UTC
Created attachment 33087 [details]
pure awesomeness of this feature showcased.

Adding a screenshot of this feature to make the intent more clear.
Comment 3 Olli-Antti Kivilahti 2014-10-31 16:04:32 UTC Comment hidden (obsolete)
Comment 4 Olli-Antti Kivilahti 2014-10-31 16:12:42 UTC Comment hidden (obsolete)
Comment 5 Olli-Antti Kivilahti 2014-11-07 14:49:24 UTC Comment hidden (obsolete)
Comment 6 Olli-Antti Kivilahti 2014-11-07 14:51:58 UTC Comment hidden (obsolete)
Comment 7 Olli-Antti Kivilahti 2014-11-07 14:57:21 UTC Comment hidden (obsolete)
Comment 8 Olli-Antti Kivilahti 2014-11-07 14:59:30 UTC Comment hidden (obsolete)
Comment 9 Olli-Antti Kivilahti 2014-11-07 15:00:32 UTC
This feature is pending proper unit tests but looking forward to some feedback from the community.
Comment 10 Koha Team University Lyon 3 2014-12-04 15:50:06 UTC
Hi,
I'll try to test this patch next thursday (testing party)
But now, patch doesn't apply on a sandbox :

The sandbox you've requested is not ready.
Some problems occurred applying patches from bug 13182:
<h1>Something went wrong !</h1>Applying: Bug 13182 - When placing an order from a staged file, check for duplicates, display availability information.
fatal: sha1 information is lacking or useless (acqui/addorderiso2709.pl).
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.
Patch failed at 0001 Bug 13182 - When placing an order from a staged file, check for duplicates, display availability information.
When you have resolved this problem run git bz apply --continue.
If you would prefer to skip this patch, instead run git bz apply --skip.
To restore the original branch and stop patching run git bz apply --abort.
Bug 13182 - When placing an order from a staged file, check for duplicates, display availability information, choose to overlay matched records.

33376 - Bug 13182 - When placing an order from a staged file, check for duplicates, display availability information.
33378 - Bug 13182 - Overlay an existing Biblio with the acquired import_record
33379 - Bug 13182 - Remove really dead code.
Comment 11 Mark Tompsett 2015-04-18 03:56:24 UTC
Created attachment 38113 [details] [review]
Bug 13182 - When placing an order from a staged file, check for duplicates, display availability information.

When making acquisitions from one staged file, by many branches as a joint effort, it is important to be able to
make orders from the same import_record and to know how many Items have already been ordered.

Also introduces misc/maintenance/matchAllStagedRecords.pl to recreate import_record matchings based on the given
matcher.

This places some extra requirements.
1. An status = 'imported' import_record needs to be shown in the stage_file so other branches can order from it as
   well. So we can't "hide" a import_record from the staged_file after an order is placed.
2. Having imported one import_record, that import_record must know it is matched to the just imported Biblio,
   so next acquirers know that it is already in our catalog.
3. We need to know which other branches have already ordered the item so we wont end up ordering too many copies
   of one Biblio to the consortium. Thus we color each row and show the count of Items in the current user's
   branch and the total amount of Items in the whole consortium. Additional tooltip listing of Item counts by
   Branch is displayed so we know are there any such Items near us.

TEST PLAN:

SETUP:

0a. Define a Matcher from admin/matching-rules.pl. We think good starting values are:
Search index,         Tag , score
ISBN,                 020a, 1000
Identifier-standard,  024a, 1000
Identifier-standard,  028a, 1000
Title,                245a, 500
Title-uniform,        240a, 500
Personal-name,        110a, 300
Corporate-name,       100a, 300
Matching threshold 800
    You can use an existing matcher as well if you have it configured to other than ISBN.

0b. Stage a MARC file.

BASIC USAGE:

1. Start making an order from a staged_file, stop when you see the import_records after you selected the batch
   to order from at
   acqui/addorderiso2709.pl?import_batch_id=50...
2. If you have matches, they are colored and the Item counts are visible. If not, order any item and return to
   the previous view and reload.
3. Observe that you should now have atleast one match. The match displays the amount of items in your branch,
   in your consortium, and a tooltip from the record title, showing a breakdown of items by branch.
4. When you add more orders, you get more order lines, but all orders are for the same Biblio, thus you don't
   get many Biblios for many orderlines.

UNLEASH THE HOUNDS OF ACQUISITIONS:

1. Order a bunch of biblios and items for homebranch A. Make sure that you get the ordered orderlines in your
   basket.
2. Order a bunch of biblios and items for homebranch B.
3. Order a bunch of biblios and items for homebranch C.
4. Observe that the biblios already ordered into your catalogue are colored and a number is in front of the
   title. First number is the count of Items for this Biblio in user's logged-in-branch, second is the total
   number of Items in the catalogue. Hovering mouse pointer over the title displays a tooltip with a breakdown
   of all Items by homebranch.

TODO: Unit tests.

NOTE: Please check rebase I did.
Comment 12 Mark Tompsett 2015-04-18 03:56:36 UTC
Created attachment 38114 [details] [review]
Bug 13182 - Overlay an existing Biblio with the acquired import_record

If we know we already have a matching biblio in Koha, we can optionally overlay the existing Biblio with the
new one. (We don't really need it but looks like Bg_12074 needs it).

TEST PLAN

1. Add Items to an order from a staged file.
2. Go and edit the newly added Biblio. Set the field 003 to "TEST ME GENTLY".
3. Add more Items for this same biblio from the same staged_file.
4. While selecting what to add, check the "Overlay?"-checkbox to replace the existing Bibliographic record
   with the import_record you are ordering.
5. Save the orderline selection.
6. Observe that the existing modified biblio has changed to it's latest revision.
Comment 13 Mark Tompsett 2015-04-18 03:56:53 UTC
Created attachment 38115 [details] [review]
Bug 13182 - Remove really dead code.

NOTE: Please check the rebase I did.
Comment 14 Mark Tompsett 2015-04-18 03:58:18 UTC
Dumping this back to Assigned, since there are admittedly lacking tests. But this should apply now, and hopefully I didn't make any typographical errors. :)
Comment 15 Olli-Antti Kivilahti 2015-04-20 08:07:10 UTC
Thanks for the rebase M. Tompsett!