Bug 39296

Summary: Provide a template plugin to return MARC::Record for MARCXML metadata
Product: Koha Reporter: David Cook <dcook>
Component: ReportsAssignee: David Cook <dcook>
Status: Signed Off --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: flyingendpaper, phil
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Attachments: Bug 39296: Provide a template plugin for getting a MARC::Record from MARCXML
Bug 39296: Provide a template plugin for getting a MARC::Record from MARCXML

Description David Cook 2025-03-11 03:39:29 UTC
ExtractValue() is a useful MySQL function but it's limited in what it can do. 

Sometimes, you want to do reporting off a more nuanced understanding of MARC.

I had thought about exposing XML::LibXML::Document and XML::LibXML::XPathContext objects, but they're too powerful/vulnerable to abuse. Instead, I decided that MARC::Record should give all the power we need while not upsetting the status quo or security (after all, we already allow access to MARC::Record objects via $biblio objects in Templates).

In this case, you fetch the whole "metadata" blob from the database in your report, and you transform it using a template using Koha::Template::Plugin::KohaMarcRecord

It lets you perform more advanced reporting using the full MARC record.
Comment 1 David Cook 2025-03-11 04:26:18 UTC
Created attachment 179135 [details] [review]
Bug 39296: Provide a template plugin for getting a MARC::Record from MARCXML

This change takes MARCXML data (e.g. from the biblio_metadata table) and
returns a MARC::Record object which can be used in Reports templates to
allow for more sophisticated reporting using MARCXML data.

Test plan:
0. Apply the patch and koha-plack --restart kohadev
1. Go to http://localhost:8081/cgi-bin/koha/tools/letter.pl
2. Add a new notice for Reports with the following content:
[% USE KohaMarcRecord %]
[% SET all_subjects = {} %]
[% FOREACH row IN data %]
    [% SET record= KohaMarcRecord.from_xml(row.metadata) %]
    [% SET fields = record.field('6..') %]
    [% FOREACH field IN fields %]
        [% tmp = field.delete_subfield('code','9') %]
        [% SET subject = field.as_string() %]
        [% IF ( all_subjects.$subject ) %]
            [% all_subjects.$subject = all_subjects.$subject + 1 %]
        [% ELSE %]
            [% all_subjects.$subject = 1 %]
        [% END %]
    [% END %]
[% END %]
<table>
<tr><th>Subject</th><th>Count</th></tr>
[% FOREACH key IN all_subjects.nsort.reverse %]
<tr><td>[% key %]</td><td>[% all_subjects.$key %]</td></tr>
[% END %]
</table>
3. Go to http://localhost:8081/cgi-bin/koha/reports/guided_reports.pl?op=add_form_sql
4. Create a new report with the following SQL:
5. Click "Run with template" and choose the template/notice you've created above
6. Note that you get a list of MARC subjects sorted by the most used to the least used
Comment 2 Phil Ringnalda 2025-03-12 01:56:46 UTC
But if we do that, all my knowledge about how to ExtractValue repeatable fields like 650 as individuals with datafield[@tag=650][1] ... datafield[@tag=650][20] will be useless!
Comment 3 Phil Ringnalda 2025-03-12 01:58:38 UTC
Created attachment 179189 [details] [review]
Bug 39296: Provide a template plugin for getting a MARC::Record from MARCXML

This change takes MARCXML data (e.g. from the biblio_metadata table) and
returns a MARC::Record object which can be used in Reports templates to
allow for more sophisticated reporting using MARCXML data.

Test plan:
0. Apply the patch and koha-plack --restart kohadev
1. Go to http://localhost:8081/cgi-bin/koha/tools/letter.pl
2. Add a new notice for Reports with the following content:
[% USE KohaMarcRecord %]
[% SET all_subjects = {} %]
[% FOREACH row IN data %]
    [% SET record= KohaMarcRecord.from_xml(row.metadata) %]
    [% SET fields = record.field('6..') %]
    [% FOREACH field IN fields %]
        [% tmp = field.delete_subfield('code','9') %]
        [% SET subject = field.as_string() %]
        [% IF ( all_subjects.$subject ) %]
            [% all_subjects.$subject = all_subjects.$subject + 1 %]
        [% ELSE %]
            [% all_subjects.$subject = 1 %]
        [% END %]
    [% END %]
[% END %]
<table>
<tr><th>Subject</th><th>Count</th></tr>
[% FOREACH key IN all_subjects.nsort.reverse %]
<tr><td>[% key %]</td><td>[% all_subjects.$key %]</td></tr>
[% END %]
</table>
3. Go to http://localhost:8081/cgi-bin/koha/reports/guided_reports.pl?op=add_form_sql
4. Create a new report with the following SQL:
select metadata from biblio_metadata
5. Click "Run with template" and choose the template/notice you've created above
6. Note that you get a list of MARC subjects sorted by the most used to the least used

Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Comment 4 Phil Ringnalda 2025-03-12 01:59:25 UTC
I made a guess at the missing SQL and added it to the test plan.
Comment 5 David Cook 2025-03-12 02:06:26 UTC
(In reply to Phil Ringnalda from comment #4)
> I made a guess at the missing SQL and added it to the test plan.

Thanks, Phil. Not sure what happened there!
Comment 6 David Cook 2025-03-12 02:57:49 UTC
(In reply to Phil Ringnalda from comment #2)
> But if we do that, all my knowledge about how to ExtractValue repeatable
> fields like 650 as individuals with datafield[@tag=650][1] ...
> datafield[@tag=650][20] will be useless!

You delight me, Phil. Haha.