Bug 32700 - Action logs logging should be divided into smaller components to reduce table size
Summary: Action logs logging should be divided into smaller components to reduce table...
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Database (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-01-23 14:37 UTC by Lari Strand
Modified: 2023-02-03 13:15 UTC (History)
1 user (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Lari Strand 2023-01-23 14:37:49 UTC
In Koha-Suomi our action_logs tables fill up with tens of millions rows of data in one year. For example our Vaski-libraries had accumulated 800 000 entries in one week.

To be able to manage this massive amount of data (cleanups/archiving) better, Koha should save the actions to dedicated tables based on interface for example. Currently we archive the action_logs table each year to archive tables based on the year found on timestamps. If the data was split into smaller tables based on interface we could better organize these cleanups.

The time it takes to make archived tables and reinserting newest entries back to a clean active action_logs table would take far less time with invidual action_log tables based on interface (action_logs_api etc.).
Also deciding the time of storage might be easier based on interface.

Moving millions of rows with massive time taking insert queries takes almost up to a day for us and would eventually bring Koha to a halt.

Ps. After we moved to version 21.11 last year, the amount of data written to action_logs has increased dramatically due to various logging settings we want to have enabled.
Comment 1 Lari Strand 2023-02-03 10:21:30 UTC
At least we should only log the changes made to biblios and items when they have been modified. In 21.11 modifying a biblio only logged all the biblio data before the modification. On item modification only the whole item with new item data is logged. This should be fixed so that only the old value and the changed new value/values gets logged. This would reduce data that is logged.