Bug 40856 - Improve Standard backend metadata retrieval
Summary: Improve Standard backend metadata retrieval
Status: Pushed to main
Alias: None
Product: Koha
Classification: Unclassified
Component: ILL (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement
Assignee: Tomás Cohen Arazi (tcohen)
QA Contact: Martin Renvoize (ashimema)
URL:
Keywords:
Depends on: 40855
Blocks: 38341
  Show dependency treegraph
 
Reported: 2025-09-22 19:22 UTC by Tomás Cohen Arazi (tcohen)
Modified: 2025-09-29 20:08 UTC (History)
4 users (show)

See Also:
GIT URL:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
25.11.00
Circulation function:


Attachments
Bug 40856: Add unit tests for Standard backend metadata method (3.78 KB, patch)
2025-09-22 19:30 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 40856: Optimize Standard backend metadata method with database-level filtering (2.32 KB, patch)
2025-09-22 19:30 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 40856: Add unit tests for Standard backend metadata method (3.84 KB, patch)
2025-09-22 20:56 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review
Bug 40856: Optimize Standard backend metadata method with database-level filtering (2.38 KB, patch)
2025-09-22 20:56 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Tomás Cohen Arazi (tcohen) 2025-09-22 19:22:21 UTC
The metadata() method in the Standard ILL backend uses an inefficient pattern for filtering attributes, performing a linear search through an ignore list for every attribute using grep inside a while loop. This approach becomes increasingly slow as the number of attributes grows, and manually iterates through all attributes instead of leveraging database-level filtering.
Comment 1 Tomás Cohen Arazi (tcohen) 2025-09-22 19:30:04 UTC
Created attachment 186742 [details] [review]
Bug 40856: Add unit tests for Standard backend metadata method

This patch adds comprehensive unit tests for the metadata() method in the
Standard ILL backend to establish baseline behavior before optimization.

The tests cover:
- Metadata structure (returns hashref)
- Included attributes (title, author, isbn, year, custom fields)
- Excluded attributes (ignored fields like requested_partners, type, etc.)
- Display name transformation (attribute types become display names)

Current behavior tested:
- metadata() filters out predefined ignore list attributes
- Uses linear search with grep for each attribute
- Transforms attribute types to display names (title -> Title)
- Custom fields get ucfirst transformation (custom_field -> Custom_field)
- Returns hashref with display names as keys

These tests document the current inefficient implementation and will
ensure the upcoming optimization maintains compatibility.

Test plan:
1. Apply patch
2. Run:
   $ ktd --shell
  k$ prove t/db_dependent/Koha/ILL/Backend/Standard.t
=> SUCCESS: Tests pass! Current metadata behavior documented
3. Tests establish baseline for upcoming optimization
4. Sign off :-D
Comment 2 Tomás Cohen Arazi (tcohen) 2025-09-22 19:30:07 UTC
Created attachment 186743 [details] [review]
Bug 40856: Optimize Standard backend metadata method with database-level filtering

This patch optimizes the metadata() method in the Standard ILL backend
to use database-level filtering instead of inefficient linear searches.

Performance improvement:
- Before: Load all attributes, then filter each with linear search
- After: Load only relevant attributes using database index

Test plan:
1. Apply patch
2. Run:
   $ ktd --shell
  k$ prove t/db_dependent/Koha/ILL/Backend/Standard.t
=> SUCCESS: Tests pass! Optimization maintains compatibility
3. Test ILL request metadata display:
   - View ILL request details
   - Verify metadata shows correctly
   - Check ignored fields are excluded
=> SUCCESS: Same functionality, better performance
4. Sign off :-D
Comment 3 Martin Renvoize (ashimema) 2025-09-22 20:56:47 UTC
Created attachment 186760 [details] [review]
Bug 40856: Add unit tests for Standard backend metadata method

This patch adds comprehensive unit tests for the metadata() method in the
Standard ILL backend to establish baseline behavior before optimization.

The tests cover:
- Metadata structure (returns hashref)
- Included attributes (title, author, isbn, year, custom fields)
- Excluded attributes (ignored fields like requested_partners, type, etc.)
- Display name transformation (attribute types become display names)

Current behavior tested:
- metadata() filters out predefined ignore list attributes
- Uses linear search with grep for each attribute
- Transforms attribute types to display names (title -> Title)
- Custom fields get ucfirst transformation (custom_field -> Custom_field)
- Returns hashref with display names as keys

These tests document the current inefficient implementation and will
ensure the upcoming optimization maintains compatibility.

Test plan:
1. Apply patch
2. Run:
   $ ktd --shell
  k$ prove t/db_dependent/Koha/ILL/Backend/Standard.t
=> SUCCESS: Tests pass! Current metadata behavior documented
3. Tests establish baseline for upcoming optimization
4. Sign off :-D

Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Comment 4 Martin Renvoize (ashimema) 2025-09-22 20:56:49 UTC
Created attachment 186761 [details] [review]
Bug 40856: Optimize Standard backend metadata method with database-level filtering

This patch optimizes the metadata() method in the Standard ILL backend
to use database-level filtering instead of inefficient linear searches.

Performance improvement:
- Before: Load all attributes, then filter each with linear search
- After: Load only relevant attributes using database index

Test plan:
1. Apply patch
2. Run:
   $ ktd --shell
  k$ prove t/db_dependent/Koha/ILL/Backend/Standard.t
=> SUCCESS: Tests pass! Optimization maintains compatibility
3. Test ILL request metadata display:
   - View ILL request details
   - Verify metadata shows correctly
   - Check ignored fields are excluded
=> SUCCESS: Same functionality, better performance
4. Sign off :-D

Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Comment 5 Martin Renvoize (ashimema) 2025-09-22 20:58:30 UTC
Simple internal API improvement, well covered by tests.

Straight for QA, Passed.
Comment 6 Lucas Gass (lukeg) 2025-09-29 20:08:03 UTC
Nice work everyone!

Pushed to main for 25.11