Bug 30069 - Add edifact_messages to cleanup_database.pl
Summary: Add edifact_messages to cleanup_database.pl
Status: Needs documenting
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Matt Blenkinsop
QA Contact: Kyle M Hall
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-02-10 13:18 UTC by Martin Renvoize
Modified: 2023-03-08 13:16 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.05.00


Attachments
Bug 30069: Add edifact-messages to cleanup_database.pl (5.96 KB, patch)
2023-02-15 14:26 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 30069: Add edifact-messages to cleanup_database.pl (5.99 KB, patch)
2023-02-15 15:55 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 30069: Add edifact-messages to cleanup_database.pl (6.02 KB, patch)
2023-02-20 21:30 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 30069: (QA follow-up) Fix capitalization for edifact (2.00 KB, patch)
2023-02-20 21:31 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 30069: (QA follow-up) Rebase and add filter (1.61 KB, patch)
2023-03-01 15:17 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 30069: Add edifact-messages to cleanup_database.pl (9.64 KB, patch)
2023-03-02 13:00 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 30069: (QA follow-up) Fix capitalization for edifact (2.08 KB, patch)
2023-03-02 13:00 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 30069: (QA follow-up) Rebase and add filter (1.67 KB, patch)
2023-03-02 13:00 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 30069: (QA follow-up) Use pure DBIC for select and delete with single query (3.54 KB, patch)
2023-03-02 13:00 UTC, Kyle M Hall
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Renvoize 2022-02-10 13:18:51 UTC
We should add the ability to clean up old edifact messages with cleanup database.
Comment 1 Matt Blenkinsop 2023-02-15 14:26:05 UTC
Created attachment 146690 [details] [review]
Bug 30069: Add edifact-messages to cleanup_database.pl

This patch allows users to clear out old edifact_messages using the cleanup_database script. The number of days can either be set in the CLI or the default value of 365 can be used.

Test plan:
1) Ensure you have some EDI orders or even just some dummy messages in the edifact_messages table
2) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose (Change the number of days according to the data in your table)
3) The response should show a number of messages that would have been deleted
4) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose --confirm
5) The response should now show the same number of messages have been deleted
6) Check your edifact_messages table to confirm that the data has been deleted
Comment 2 Matt Blenkinsop 2023-02-15 15:55:33 UTC
Created attachment 146694 [details] [review]
Bug 30069: Add edifact-messages to cleanup_database.pl

This patch allows users to clear out old edifact_messages using the cleanup_database script. The number of days can either be set in the CLI or the default value of 365 can be used.

Test plan:
1) Ensure you have some EDI orders or even just some dummy messages in the edifact_messages table
2) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose (Change the number of days according to the data in your table)
3) The response should show a number of messages that would have been deleted
4) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose --confirm
5) The response should now show the same number of messages have been deleted
6) Check your edifact_messages table to confirm that the data has been deleted

Sponsored-by: PTFS Europe
Comment 3 Katrin Fischer 2023-02-20 21:30:57 UTC
Created attachment 146976 [details] [review]
Bug 30069: Add edifact-messages to cleanup_database.pl

This patch allows users to clear out old edifact_messages using the cleanup_database script. The number of days can either be set in the CLI or the default value of 365 can be used.

Test plan:
1) Ensure you have some EDI orders or even just some dummy messages in the edifact_messages table
2) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose (Change the number of days according to the data in your table)
3) The response should show a number of messages that would have been deleted
4) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose --confirm
5) The response should now show the same number of messages have been deleted
6) Check your edifact_messages table to confirm that the data has been deleted

Sponsored-by: PTFS Europe
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 4 Katrin Fischer 2023-02-20 21:31:00 UTC
Created attachment 146977 [details] [review]
Bug 30069: (QA follow-up) Fix capitalization for edifact

Changes a few occurrences of edifact in the output messages
to EDIFACT.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 5 Katrin Fischer 2023-02-20 21:32:44 UTC
I have no experience with the EDIFACT process, so I am not sure what role the EDIFACT messages play. Could this cause any issues if the orders haven't been received yet?
Comment 6 Kyle M Hall 2023-02-21 11:59:38 UTC
(In reply to Katrin Fischer from comment #5)
> I have no experience with the EDIFACT process, so I am not sure what role
> the EDIFACT messages play. Could this cause any issues if the orders haven't
> been received yet?

Yes, that's a good thought. I think the query should but updated to not delete messages marked as 'new'. If automatic invoice importing is disabled, this feature should delete un-imported EDI invoices!
Comment 7 Matt Blenkinsop 2023-03-01 15:17:01 UTC
Created attachment 147564 [details] [review]
Bug 30069: (QA follow-up) Rebase and add filter

This patch now checks the status of messages and ignores any message
with a status of 'new'. It is also rebased to account for changes made
to cleanup_database.pl in bug 17350.

Test plan:
1) Ensure you have some EDI orders or even just some dummy messages in the edifact_messages table with a mixture of statuses including 'new'
2) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose (Change the number of days according to the data in your table)
3) The response should show a number of messages that would have been deleted
4) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose --confirm
5) The response should now show the same number of messages have been deleted
6) Check your edifact_messages table to confirm that the data has been deleted
7) Confirm that no messages marked 'new' have been deleted
Comment 8 Matt Blenkinsop 2023-03-01 15:18:28 UTC
Patch has been rebased and the requested check for 'new' messages has been added. Kyle I'm not too sure I understand what you are asking for in the second part of your comment about un-imported invoices, could you clarify further? Thanks
Comment 9 Kyle M Hall 2023-03-02 11:47:26 UTC
(In reply to Matt Blenkinsop from comment #8)
> Patch has been rebased and the requested check for 'new' messages has been
> added. Kyle I'm not too sure I understand what you are asking for in the
> second part of your comment about un-imported invoices, could you clarify
> further? Thanks

Invoices may not be processed and imported as soon as they are downloaded. It depends on the syspref  EdifactInvoiceImport
Comment 10 Kyle M Hall 2023-03-02 13:00:05 UTC
Created attachment 147606 [details] [review]
Bug 30069: Add edifact-messages to cleanup_database.pl

This patch allows users to clear out old edifact_messages using the cleanup_database script. The number of days can either be set in the CLI or the default value of 365 can be used.

Test plan:
1) Ensure you have some EDI orders or even just some dummy messages in the edifact_messages table
2) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose (Change the number of days according to the data in your table)
3) The response should show a number of messages that would have been deleted
4) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose --confirm
5) The response should now show the same number of messages have been deleted
6) Check your edifact_messages table to confirm that the data has been deleted

Sponsored-by: PTFS Europe
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 11 Kyle M Hall 2023-03-02 13:00:13 UTC
Created attachment 147607 [details] [review]
Bug 30069: (QA follow-up) Fix capitalization for edifact

Changes a few occurrences of edifact in the output messages
to EDIFACT.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 12 Kyle M Hall 2023-03-02 13:00:15 UTC
Created attachment 147608 [details] [review]
Bug 30069: (QA follow-up) Rebase and add filter

This patch now checks the status of messages and ignores any message
with a status of 'new'. It is also rebased to account for changes made
to cleanup_database.pl in bug 17350.

Test plan:
1) Ensure you have some EDI orders or even just some dummy messages in the edifact_messages table with a mixture of statuses including 'new'
2) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose (Change the number of days according to the data in your table)
3) The response should show a number of messages that would have been deleted
4) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose --confirm
5) The response should now show the same number of messages have been deleted
6) Check your edifact_messages table to confirm that the data has been deleted
7) Confirm that no messages marked 'new' have been deleted

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 13 Kyle M Hall 2023-03-02 13:00:18 UTC
Created attachment 147609 [details] [review]
Bug 30069: (QA follow-up) Use pure DBIC for select and delete with single query

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 14 Tomás Cohen Arazi 2023-03-02 14:59:29 UTC
Pushed to master for 23.05.

Nice work everyone, thanks!
Comment 15 Matt Blenkinsop 2023-03-03 13:35:09 UTC
Not backporting to 22.11.x - enhancement