Bug 21272 - MARC import should warn about mis-matched branch and item-type fields in 952 during staged import
Summary: MARC import should warn about mis-matched branch and item-type fields in 952 ...
Status: Signed Off
Alias: None
Product: Koha
Classification: Unclassified
Component: Tools (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Hammat wele
QA Contact: Testopia
URL:
Keywords:
: 14768 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-08-26 16:35 UTC by Barton Chittenden
Modified: 2023-11-09 20:59 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
This enhancement adds a warning message when importing staged records to show library codes in the imported file that do not match with what is configured in Koha. It also does not import these items.
Version(s) released in:


Attachments
Bug 21272: MARC import should warn about mis-matched branch and item-type fields in 952 during staged import (8.38 KB, patch)
2023-08-31 19:16 UTC, Hammat wele
Details | Diff | Splinter Review
Bug 21272: Unit test (2.63 KB, patch)
2023-08-31 19:16 UTC, Hammat wele
Details | Diff | Splinter Review
Exported Marc file that contains items with branche code (22.03 KB, application/marc)
2023-08-31 19:39 UTC, Hammat wele
Details
Bug 21272: MARC import should warn about mis-matched branch and item-type fields in 952 during staged import (8.38 KB, patch)
2023-09-12 03:35 UTC, David Nind
Details | Diff | Splinter Review
Bug 21272: Unit test (2.62 KB, patch)
2023-09-12 03:35 UTC, David Nind
Details | Diff | Splinter Review
Bug 21272: (follow-up) Terminology and formatting (1.58 KB, patch)
2023-09-12 04:05 UTC, David Nind
Details | Diff | Splinter Review
Bug 21272: (follow-up) using List::MoreUtils#any to get rid of %branches and fixing GetImportItemsBranches in ImportBatch.pm (4.34 KB, patch)
2023-11-09 00:28 UTC, Hammat wele
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Barton Chittenden 2018-08-26 16:35:32 UTC
When importing a record that has homebranch (952$a), holding branch (952$b) or item type (952$y) populated, but not matching values in branches.branchcode or itemtypes.itemtype, koha will indicate that items were not imported (via the count of imported items), but not why.

This causes frequent confusion among new users. It would be nice to offer mappings between detected branches / item types and the branches and item types already in the system.
Comment 1 Katrin Fischer 2023-08-26 19:57:46 UTC
*** Bug 14768 has been marked as a duplicate of this bug. ***
Comment 2 Hammat wele 2023-08-31 19:16:07 UTC
Created attachment 155038 [details] [review]
Bug 21272: MARC import should warn about mis-matched branch and item-type fields in 952 during staged import

This plugin add a warning message to show missing branches during staged import, it also ignore item with wrong banches.

To test

Before make sure backgroundjobs process is running (/misc/workers/background_jobs_worker.pl --queue long_tasks)

1- Go to Cataloging -> Stage records for import
2 - Upload a MARC file (koha.mrc) that contain items with homebranch and/or homelibraary set and is not present in your installation
3- Click on Stage for import
4- Click on View batch
    --> There is no message for the missing branches
5- Click on "Import this batch into the catalog"
    --> The importation failed
6- Apply the patch
7- Run prove -t t/db_dependent/ImportBatch.t
8- Click on "Manage imported batch"
    --> There is a warning message showing the missing branches in a link
9- Click on "Import this batch into the catalog" again
    --> The records are imported
    --> The items with wrong branches are ignored

https://bugs.koha-community.org/show_bug.cgi?id=12532
Comment 4 Hammat wele 2023-08-31 19:39:33 UTC
Created attachment 155043 [details]
Exported Marc file that contains items with branche code
Comment 5 David Nind 2023-09-12 03:35:29 UTC
Created attachment 155523 [details] [review]
Bug 21272: MARC import should warn about mis-matched branch and item-type fields in 952 during staged import

This plugin add a warning message to show missing branches during staged import, it also ignore item with wrong banches.

To test

Before make sure backgroundjobs process is running (/misc/workers/background_jobs_worker.pl --queue long_tasks)

1- Go to Cataloging -> Stage records for import
2 - Upload a MARC file (koha.mrc) that contain items with homebranch and/or homelibraary set and is not present in your installation
3- Click on Stage for import
4- Click on View batch
    --> There is no message for the missing branches
5- Click on "Import this batch into the catalog"
    --> The importation failed
6- Apply the patch
7- Run prove -t t/db_dependent/ImportBatch.t
8- Click on "Manage imported batch"
    --> There is a warning message showing the missing branches in a link
9- Click on "Import this batch into the catalog" again
    --> The records are imported
    --> The items with wrong branches are ignored

Signed-off-by: David Nind <david@davidnind.com>
Comment 6 David Nind 2023-09-12 03:35:32 UTC
Created attachment 155524 [details] [review]
Bug 21272: Unit test

Signed-off-by: David Nind <david@davidnind.com>
Comment 7 David Nind 2023-09-12 03:59:14 UTC
Thanks for adding a sample MARC file!

I amended the commit messages to remove the link to bug 12532 (hope I have done this correctly!).

I will add a follow-up to fix terminology and formatting:
- In US English for the warning message: no space between the end of the warning message and the colon
- "...missing branches...": changed to "...missing libraries..." (see https://wiki.koha-community.org/wiki/Terminology#B)

For me, this did not seem to show a warning message when the item types in 952 don't match. I'm assuming these patches only cover library code mismatches.
Comment 8 David Nind 2023-09-12 04:05:56 UTC
Created attachment 155525 [details] [review]
Bug 21272: (follow-up) Terminology and formatting

Change warning message to "..missing libraries...".

Remove space before colon for the warning message.

Signed-off-by: David Nind <david@davidnind.com>
Comment 9 Kyle M Hall 2023-10-27 10:53:55 UTC
Comment on attachment 155523 [details] [review]
Bug 21272: MARC import should warn about mis-matched branch and item-type fields in 952 during staged import

Review of attachment 155523 [details] [review]:
-----------------------------------------------------------------

::: C4/ImportBatch.pm
@@ +819,5 @@
> +            my ( $holdingbranchfield, $holdingbranchsubfield ) = GetMarcFromKohaField('items.holdingbranch');
> +            my $holdingbranch = $item_marc->subfield( $holdingbranchfield, $holdingbranchsubfield );
> +
> +            my @branches = map { $_->branchcode } Koha::Libraries->search->as_list;
> +            my %branches = map { $_ => 1 } @branches;

why not use https://perldoc.perl.org/List::Util#any and get rid of %branches

@@ +1235,5 @@
> +        WHERE import_batch_id = ?
> +        ORDER BY import_items_id";
> +    my @params;
> +    push( @params, $batch_id );
> +    my $sth = $dbh->prepare_cached($query);

Should this be a cached querey?
Comment 10 Hammat wele 2023-11-09 00:28:01 UTC
Created attachment 158683 [details] [review]
Bug 21272: (follow-up) using List::MoreUtils#any to get rid of %branches and fixing GetImportItemsBranches in ImportBatch.pm
Comment 11 Katrin Fischer 2023-11-09 20:59:28 UTC
Please also fix the bug description, I think this bug only covers one half of it.