Bug 34355 - Automated MARC record ordering process
Summary: Automated MARC record ordering process
Status: ASSIGNED
Alias: None
Product: Koha
Classification: Unclassified
Component: Acquisitions (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement with 10 votes (vote)
Assignee: Matt Blenkinsop
QA Contact: Testopia
URL:
Keywords:
Depends on: 35026 33170
Blocks:
  Show dependency treegraph
 
Reported: 2023-07-24 13:48 UTC by Matt Blenkinsop
Modified: 2024-03-20 19:05 UTC (History)
11 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 34355: DB schema and atomic update file (13.01 KB, patch)
2023-08-23 08:34 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Update menus (4.05 KB, patch)
2023-08-23 08:34 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add account creation (22.28 KB, patch)
2023-08-23 08:34 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add cronjob, MarcOrder object and refactored addorderiso2709 script (51.98 KB, patch)
2023-08-23 08:34 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Make item addition dependent on MarcItemFieldsToOrder mappings and introduce MarcFieldsToOrder to cronjob (11.59 KB, patch)
2023-08-23 08:34 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Fix form rendering issue (16.22 KB, patch)
2023-08-23 08:34 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: DB schema and atomic update file (13.01 KB, patch)
2023-10-13 13:51 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Update menus (4.04 KB, patch)
2023-10-13 13:51 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add account creation (22.28 KB, patch)
2023-10-13 13:51 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add cronjob, MarcOrder object and refactored addorderiso2709 script (42.78 KB, patch)
2023-10-13 13:51 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Make item addition dependent on MarcItemFieldsToOrder mappings and introduce MarcFieldsToOrder to cronjob (10.86 KB, patch)
2023-10-13 13:51 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Fix form rendering issue (16.22 KB, patch)
2023-10-13 13:51 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add match_field and match_value to allow multiple accounts in the same file directory (11.09 KB, patch)
2023-10-13 13:51 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add a plugin hook - before_orderline_create (1.17 KB, patch)
2023-10-13 13:51 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Rebase changes made in bug 33170 (10.49 KB, patch)
2023-10-13 13:51 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: DB schema and atomic update file (13.01 KB, patch)
2024-01-11 14:52 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Update menus (4.20 KB, patch)
2024-01-11 14:52 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add account creation (22.28 KB, patch)
2024-01-11 14:52 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add cronjob, MarcOrder object and refactored addorderiso2709 script (21.60 KB, patch)
2024-01-11 14:52 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Make item addition dependent on MarcItemFieldsToOrder mappings and introduce MarcFieldsToOrder to cronjob (11.07 KB, patch)
2024-01-11 14:52 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Fix form rendering issue (15.87 KB, patch)
2024-01-11 14:52 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add match_field and match_value to allow multiple accounts in the same file directory (11.10 KB, patch)
2024-01-11 14:52 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add a plugin hook - before_orderline_create (651 bytes, patch)
2024-01-11 14:53 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Rebase changes made in bug 33170 (1.06 KB, patch)
2024-01-11 14:53 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: DB schema and atomic update file (13.01 KB, patch)
2024-01-11 15:06 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Update menus (4.20 KB, patch)
2024-01-11 15:06 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add account creation (22.28 KB, patch)
2024-01-11 15:06 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add cronjob, MarcOrder object and refactored addorderiso2709 script (35.66 KB, patch)
2024-01-11 15:06 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Make item addition dependent on MarcItemFieldsToOrder mappings and introduce MarcFieldsToOrder to cronjob (11.07 KB, patch)
2024-01-11 15:06 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Fix form rendering issue (15.87 KB, patch)
2024-01-11 15:06 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add match_field and match_value to allow multiple accounts in the same file directory (11.10 KB, patch)
2024-01-11 15:06 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Add a plugin hook - before_orderline_create (651 bytes, patch)
2024-01-11 15:06 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 34355: Rebase changes made in bug 33170 (1.06 KB, patch)
2024-01-11 15:06 UTC, Matt Blenkinsop
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Blenkinsop 2023-07-24 13:48:41 UTC
This is a bug to create a process for automatically creating order lines from a MARC record. Users can nominate a directory for a particular vendor and then set a cronjob to periodically check that directory for new files. The cronjob will then stage the file, import biblios/items as required and then create order lines
Comment 1 Laurence Rault 2023-07-24 15:18:21 UTC
Koha has already EDI functionnality, that can do what you describe.
See https://koha-community.org/manual/23.05/en/html/acquisitions.html#edi-process
Comment 2 Jessie Zairo 2023-07-27 17:53:38 UTC
This enhancement would be for non-edi vendors and libraries who receive marc files.
Comment 3 Matt Blenkinsop 2023-08-23 08:34:24 UTC
Created attachment 154705 [details] [review]
Bug 34355: DB schema and atomic update file
Comment 4 Matt Blenkinsop 2023-08-23 08:34:26 UTC
Created attachment 154706 [details] [review]
Bug 34355: Update menus
Comment 5 Matt Blenkinsop 2023-08-23 08:34:29 UTC
Created attachment 154707 [details] [review]
Bug 34355: Add account creation
Comment 6 Matt Blenkinsop 2023-08-23 08:34:31 UTC
Created attachment 154708 [details] [review]
Bug 34355: Add cronjob, MarcOrder object and refactored addorderiso2709 script
Comment 7 Matt Blenkinsop 2023-08-23 08:34:33 UTC
Created attachment 154709 [details] [review]
Bug 34355: Make item addition dependent on MarcItemFieldsToOrder mappings and introduce MarcFieldsToOrder to cronjob
Comment 8 Matt Blenkinsop 2023-08-23 08:34:36 UTC
Created attachment 154710 [details] [review]
Bug 34355: Fix form rendering issue

This patch also fixes the referencing of import_record_id_selected
Comment 9 Katrin Fischer 2023-08-24 08:28:16 UTC
I feel like we will need a nice and concise test plan here as there already was some confusion about the functionality!
Comment 10 Matt Blenkinsop 2023-08-30 16:23:08 UTC
There is a nice, detailed (ish) test plan that I've written but I'm trying to write unit tests before submitting everything and marking as needing signoff.

I'm also weighing up Nick's point from Kohacon about splitting this into multiple bugs as there's some refactoring of the addorderiso2709.pl script at the heart of the enhancement
Comment 11 Katrin Fischer 2023-09-04 13:36:46 UTC
(In reply to Matt Blenkinsop from comment #10)
> There is a nice, detailed (ish) test plan that I've written but I'm trying
> to write unit tests before submitting everything and marking as needing
> signoff.
> 
> I'm also weighing up Nick's point from Kohacon about splitting this into
> multiple bugs as there's some refactoring of the addorderiso2709.pl script
> at the heart of the enhancement

+1 :)
Comment 12 Matt Blenkinsop 2023-10-13 13:51:14 UTC
Created attachment 157092 [details] [review]
Bug 34355: DB schema and atomic update file
Comment 13 Matt Blenkinsop 2023-10-13 13:51:16 UTC
Created attachment 157093 [details] [review]
Bug 34355: Update menus
Comment 14 Matt Blenkinsop 2023-10-13 13:51:19 UTC
Created attachment 157094 [details] [review]
Bug 34355: Add account creation
Comment 15 Matt Blenkinsop 2023-10-13 13:51:22 UTC
Created attachment 157095 [details] [review]
Bug 34355: Add cronjob, MarcOrder object and refactored addorderiso2709 script
Comment 16 Matt Blenkinsop 2023-10-13 13:51:25 UTC
Created attachment 157096 [details] [review]
Bug 34355: Make item addition dependent on MarcItemFieldsToOrder mappings and introduce MarcFieldsToOrder to cronjob
Comment 17 Matt Blenkinsop 2023-10-13 13:51:28 UTC
Created attachment 157097 [details] [review]
Bug 34355: Fix form rendering issue

This patch also fixes the referencing of import_record_id_selected
Comment 18 Matt Blenkinsop 2023-10-13 13:51:30 UTC
Created attachment 157098 [details] [review]
Bug 34355: Add match_field and match_value to allow multiple accounts in the same file directory
Comment 19 Matt Blenkinsop 2023-10-13 13:51:33 UTC
Created attachment 157099 [details] [review]
Bug 34355: Add a plugin hook - before_orderline_create
Comment 20 Matt Blenkinsop 2023-10-13 13:51:35 UTC
Created attachment 157100 [details] [review]
Bug 34355: Rebase changes made in bug 33170
Comment 21 Katrin Fischer 2023-10-13 18:02:03 UTC
We need a test plan please.
Comment 22 Katrin Fischer 2023-10-13 18:03:16 UTC
(In reply to Katrin Fischer from comment #21)
> We need a test plan please.

... and in general please more commit messages  ;)
Comment 23 Matt Blenkinsop 2024-01-11 14:52:40 UTC
Created attachment 160858 [details] [review]
Bug 34355: DB schema and atomic update file
Comment 24 Matt Blenkinsop 2024-01-11 14:52:43 UTC
Created attachment 160859 [details] [review]
Bug 34355: Update menus
Comment 25 Matt Blenkinsop 2024-01-11 14:52:46 UTC
Created attachment 160860 [details] [review]
Bug 34355: Add account creation
Comment 26 Matt Blenkinsop 2024-01-11 14:52:49 UTC
Created attachment 160861 [details] [review]
Bug 34355: Add cronjob, MarcOrder object and refactored addorderiso2709 script
Comment 27 Matt Blenkinsop 2024-01-11 14:52:52 UTC
Created attachment 160862 [details] [review]
Bug 34355: Make item addition dependent on MarcItemFieldsToOrder mappings and introduce MarcFieldsToOrder to cronjob
Comment 28 Matt Blenkinsop 2024-01-11 14:52:55 UTC
Created attachment 160863 [details] [review]
Bug 34355: Fix form rendering issue

This patch also fixes the referencing of import_record_id_selected
Comment 29 Matt Blenkinsop 2024-01-11 14:52:57 UTC
Created attachment 160864 [details] [review]
Bug 34355: Add match_field and match_value to allow multiple accounts in the same file directory
Comment 30 Matt Blenkinsop 2024-01-11 14:53:00 UTC
Created attachment 160865 [details] [review]
Bug 34355: Add a plugin hook - before_orderline_create
Comment 31 Matt Blenkinsop 2024-01-11 14:53:03 UTC
Created attachment 160866 [details] [review]
Bug 34355: Rebase changes made in bug 33170
Comment 32 Matt Blenkinsop 2024-01-11 15:06:21 UTC
Created attachment 160867 [details] [review]
Bug 34355: DB schema and atomic update file
Comment 33 Matt Blenkinsop 2024-01-11 15:06:25 UTC
Created attachment 160868 [details] [review]
Bug 34355: Update menus
Comment 34 Matt Blenkinsop 2024-01-11 15:06:27 UTC
Created attachment 160869 [details] [review]
Bug 34355: Add account creation
Comment 35 Matt Blenkinsop 2024-01-11 15:06:30 UTC
Created attachment 160870 [details] [review]
Bug 34355: Add cronjob, MarcOrder object and refactored addorderiso2709 script
Comment 36 Matt Blenkinsop 2024-01-11 15:06:33 UTC
Created attachment 160871 [details] [review]
Bug 34355: Make item addition dependent on MarcItemFieldsToOrder mappings and introduce MarcFieldsToOrder to cronjob
Comment 37 Matt Blenkinsop 2024-01-11 15:06:36 UTC
Created attachment 160872 [details] [review]
Bug 34355: Fix form rendering issue

This patch also fixes the referencing of import_record_id_selected
Comment 38 Matt Blenkinsop 2024-01-11 15:06:39 UTC
Created attachment 160873 [details] [review]
Bug 34355: Add match_field and match_value to allow multiple accounts in the same file directory
Comment 39 Matt Blenkinsop 2024-01-11 15:06:42 UTC
Created attachment 160874 [details] [review]
Bug 34355: Add a plugin hook - before_orderline_create
Comment 40 Matt Blenkinsop 2024-01-11 15:06:45 UTC
Created attachment 160875 [details] [review]
Bug 34355: Rebase changes made in bug 33170
Comment 41 Jessie Zairo 2024-01-19 12:52:28 UTC
Test Plan
1) Create an sFTP server (1 set up for each unique vendor) and share the credentials with the vendor who will be sending the test data
2)apply patch
3) navigate to Administration -> Acquisition parameters -> MARC order accounts
4) click New Account
5)Enter the following details
 a) Vendor:
 b) Budget:This budget will be used as the fallback value if the MARC records do not contain a mapped value for a budget code.
 c)Description:
 d)Download directory:The download directory specifies the directory in your koha installation that should be searched for new files.
 e)Match field:(Optional): If you have files from multiple vendors in the same file directory, the match field is the field in the marc record that will be checked to see if the file should be processed by this account.
 f)Match value: (Optional): This is the value that will be checked against the match field to see if the file matches this account. If it does it will be processed by this account, if not it will be skipped.
6) enter File import settings
7) enter Record matching settings
8) select if it should check for embedded item record data
9) click submit
10) drop test file in sFTP server (run manually or have a cron run to pick up the file)
11) file should load in basket for said vendor and create orders.
Comment 42 Katrin Fischer 2024-01-22 21:06:50 UTC
Is the setup of the mentioned SFTP server something done in Koha or something done outside? 
Are you expected to set it up on the Koha server itself? 
Maybe this works similar like picking up files for EDI?