Bug 29354 - Make overdue_notices.pl send HTML attachment as .html
Summary: Make overdue_notices.pl send HTML attachment as .html
Status: Needs documenting
Alias: None
Product: Koha
Classification: Unclassified
Component: Notices (show other bugs)
Version: master
Hardware: All All
: P5 - low normal (vote)
Assignee: Magnus Enger
QA Contact: Testopia
URL:
Keywords:
: 14347 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-10-28 14:35 UTC by Lucas Gass
Modified: 2023-08-05 11:49 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.05.00,22.11.04


Attachments
Bug 29354: Make overdue_notices.pl send .html (6.03 KB, patch)
2022-12-12 13:46 UTC, Magnus Enger
Details | Diff | Splinter Review
Bug 29354: Make overdue_notices.pl send .html (6.09 KB, patch)
2022-12-12 20:49 UTC, Lucas Gass
Details | Diff | Splinter Review
Bug 29354: Make overdue_notices.pl send .html (6.14 KB, patch)
2023-02-10 19:06 UTC, Kyle M Hall
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Lucas Gass 2021-10-28 14:35:26 UTC
overdue_notices.pl can output HTML data to a specified directory as an .html file but there is no way to send the overdues to the library admin email address as an attachment.html file. The file can only send as an attachment.txt.

It would be nice if we could deliver those print overdoes to librarians as .html via email.
Comment 1 Marcel de Rooy 2022-07-01 06:38:13 UTC
I opened bug 31075 for a wider scope
Comment 2 Lucas Gass 2022-07-01 13:45:51 UTC
(In reply to Marcel de Rooy from comment #1)
> I opened bug 31075 for a wider scope

Thanks Marcel, in that case should we mark this one as a duplicate of Bug 31075?
Comment 3 Marcel de Rooy 2022-07-04 06:36:25 UTC
(In reply to Lucas Gass from comment #2)
> (In reply to Marcel de Rooy from comment #1)
> > I opened bug 31075 for a wider scope
> 
> Thanks Marcel, in that case should we mark this one as a duplicate of Bug
> 31075?

I would wait a while :)
Comment 4 Magnus Enger 2022-12-09 13:14:28 UTC
I'm reclassifying this as a "normal" bug, not an enh. If you have 

- the syspref EmailOverduesNoEmail = Send and 
- specify "--html somedir" 

you will be sent a file called attachment.txt, which contains partial HTML, without html, head and body elements. This file will be pretty useless. 

I will try to provide a patch to fix this situation in a minimal way. This will not be the larger rewrite proposed in Bug 31075.
Comment 5 Magnus Enger 2022-12-12 13:46:38 UTC Comment hidden (obsolete)
Comment 6 Lucas Gass 2022-12-12 20:49:43 UTC Comment hidden (obsolete)
Comment 7 Lucas Gass 2022-12-12 20:50:15 UTC
Works as described, thanks Magnus!
Comment 8 Kyle M Hall 2023-02-10 19:06:11 UTC
Created attachment 146510 [details] [review]
Bug 29354: Make overdue_notices.pl send .html

If you have EmailOverduesNoEmail = Send and specify "--html somedir",
overdue_notices.pl will send a file by email that contains partial
HTML, as a file called attachment.txt. This patch fixes that.

To reproduce in koha-testing-docker:
- EmailOverduesNoEmail = Send
- Make sure you have a loan that was due yesterday, by backdating the
  due date
- Set up an overdue action to send "Overdue Notice" (ODUE) to the
  category you made the loan to above, when a loan is one day overdue
- Run this command:
  $ sudo koha-shell -c "perl misc/cronjobs/overdue_notices.pl -v -t -html /tmp/" kohadev
- Look at the file /tmp/notices-<DATE>.html and make sure it is a full
  HTML document, with <html>, <head>, <body> etc.
- Create a report like this:
  SELECT message_id, letter_id, borrowernumber, subject, CONCAT( '<pre>', content, '</pre>' ) AS content,
    metadata, letter_code, message_transport_type, time_queued, updated_on, to_address, content_type, failure_code
  FROM message_queue
  WHERE subject = 'Overdue Notices'
  ORDER BY message_id DESC
- Run the report and verify there is a line like this in the "content"
  of the newest message:
  Content-Type: text/plain; name=attachment.txt
- A part of the "content" will be a block of several lines of gibberish
  (base64) that look something like "RGVhciAga29oYSwNCg0KQWN...". Copy
  this block of text to somewhere like base64decode.org and decode the
  text. You should see a fragment of HTML, without <html>, <head>,
  <body> etc.

To test:
- Apply the patch
- Run overdue_notices.pl again, with the same arguments as above
- Make sure /tmp/notices-<DATE>.html is still a full HTML document
- Re-run the report, and make sure you now have this in the "content":
  Content-Type: text/html; name=attachment.html
- Decode the base64 and make sure it is now a full HTML document, with
  <html>, <head>, <body> etc.
- Re-run overdue_notices.pl as above, but replace "-html /tmp/" with
  "-csv /tmp/test.csv"
- Make sure /tmp/test.csv and the decoded base64 from the report
  contains CSV data
- Re-run overdue_notices.pl as above, but replace "-html /tmp/" with
  "-text /tmp/"
- Make sure /tmp/notices-<DATE>.txt and the decoded base64 from the
  report contains no HTML

Note:
- The actual text from the different messages will be enclosed in
  <pre>-tags
- If you have HTML in your ODUE message template and run with -v, you
  will have warnings saying "The following terms were not matched and
  replaced"
These are due to Bug 14347, and are not adressed by the current patch.

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 9 Kyle M Hall 2023-02-10 19:06:43 UTC
The file remains untouched so I don't think that requires a new command line switch.
Comment 10 Tomás Cohen Arazi 2023-03-14 12:10:22 UTC
Pushed to master for 23.05.

Nice work everyone, thanks!
Comment 11 Jacob O'Mara 2023-03-16 19:48:55 UTC
Nice work, thanks everyone!

Pushed to 22.11.x for the next release.
Comment 12 Lucas Gass 2023-04-14 18:19:36 UTC
This patchset doesn't apply cleanly in 22.05, no backport.
Comment 13 Katrin Fischer 2023-08-05 11:49:42 UTC
*** Bug 14347 has been marked as a duplicate of this bug. ***