Bug 31259

Summary: Downloading patron card PDF hangs the server
Product: Koha Reporter: Jake Deery <jake.deery>
Component: Label/patron card printingAssignee: Nick Clemens (kidclamp) <nick>
Status: RESOLVED FIXED QA Contact: Testopia <testopia>
Severity: major    
Priority: P5 - low CC: arthur.suzuki, david, dcook, lucas, madamyk, martin.renvoize, nick, wizzyrea
Version: 21.11   
Hardware: PC   
OS: Linux   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21092
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.05.00,22.11.05,22.05.12,21.11.20
Circulation function:
Attachments: screenshot of problem template
Bug 31259: Escape infinite loop when lime cannot wrap enough
Bug 31259: Escape infinite loop when lime cannot wrap enough
Bug 31259: Escape infinite loop when lime cannot wrap enough

Description Jake Deery 2022-07-29 09:09:25 UTC
Hi,

One of our customers is having an issue with their patron card creator, that we have been able to replicate on our support server. Both installations are Debian package installations, and both installations run atop Debian 10.

When one goes to create a 'patron card' using the build-in patron card creator found under Tools > Patron card creator, the download process loops endlessly and eventually returns a Proxy Timeout (we have tried without Plack).

Steps to reproduce:
1.    Create printer profile with:
      *    0mm x 0mm offset
      *    0mm x 0mm creep
2.    Create card template with:
      *    297mm x 210mm page
      *    99.1mm x 38.1 card
      *    15mm top page margin
      *    5mm left page margin
      *    2 columns
      *    7 rows
      *    3mm gap between columns
      *    0mm gap between rows
3.    Create layout with:
      *    Guide box on
      *    Guide grid off
      *    Field 1 text to 'Name: <firstname> <surname>'
      *    Font to 'Courier'
      *    Font size to 14pt
      *    Text alignment to left
      *    Lower left X co-ord to 20mm
      *    Lower left Y co-ord to 25mm
      *    Print card number as barcode: yes
      *    Lower left X co-ord: 30
      *    Lower left Y co-ord: 10
      *    Barcode type: CODE39
      *    Print card number as text under barcode: yes
4.    Add two patrons to a new card batch, and use the previously created printer profile, card template, and layout.
5.    Export the entire batch as a PDF.
6.    The software will hang, and a 'create-ptf.pl' process will run indefinitely
koha@demo1:~$ sudo ps aux | grep perl
support+ 30344 93.1  5.2 287880 210820 ?       R    10:06   0:05 /usr/bin/perl /usr/share/koha/intranet/cgi-bin/patroncards/create-pdf.pl

The create-ptf.pl script eats the CPU, and also appears to leak memory indefinitely, until the OOM killer is forced to step in. I have tried changing a few different values, including the font (as I thought maybe it was Courier being missing in Linux), all to no avail.

If anyone has the time to take a look at this, or has a vague idea of what lines to look at so I can see if I can fix it myself, that would be hugely appreciated.

Many thanks,
Jake
Comment 1 Chris Nighswonger 2022-07-31 12:26:41 UTC
Please post this URL from your instance:

http://staff-client.kohadev.org/cgi-bin/koha/patroncards/create-pdf.pl?batch_id=1&template_id=1&layout_id=5&start_label=1

Include the parameters which cause the loop.
Comment 3 David Cook 2022-08-02 00:30:49 UTC
I had the same problem years ago but fixed it with bug 22429 so must be something else causing the problem now. 

The infinite loop is probably in the "while(1)" block, and it probably relates to whatever data you're feeding in. 

Are you able to replicate this problem using koha-testing-docker?
Comment 4 Chris Nighswonger 2022-08-02 00:40:56 UTC
(In reply to Jake Deery from comment #2)
> Hi Chris,
> 
> This is the URL:
> https://support-staff.koha-ptfs.co.uk/cgi-bin/koha/patroncards/create-pdf.
> pl?batch_id=2&template_id=8&layout_id=9&layout_back_id=9&start_card=1 
> 

Could you post the results of the following query:

SELECT * FROM creator_batches WHERE batch_id=2;
Comment 5 Chris Nighswonger 2022-08-02 00:44:41 UTC
(In reply to Jake Deery from comment #2)
> Hi Chris,
> 
> This is the URL:
> https://support-staff.koha-ptfs.co.uk/cgi-bin/koha/patroncards/create-pdf.
> pl?batch_id=2&template_id=8&layout_id=9&layout_back_id=9&start_card=1 

In addition, try this URL and see if it tight-loops:

https://support-staff.koha-ptfs.co.uk/cgi-bin/koha/patroncards/create-pdf.pl?batch_id=2&template_id=8&layout_id=9&start_card=1
Comment 6 Chris Nighswonger 2022-08-02 01:00:00 UTC
(In reply to David Cook from comment #3)
> I had the same problem years ago but fixed it with bug 22429 so must be
> something else causing the problem now. 
> 
> The infinite loop is probably in the "while(1)" block, and it probably
> relates to whatever data you're feeding in.

This while block is problematic (says the one who coded this mess): it has no out in the case that the main conditional never evaluates true. There may be some fringe cases where the regexp does not match... ever.

Hard to tell without data which causes the infinite looping.
Comment 7 Chris Nighswonger 2022-08-03 01:10:36 UTC
(In reply to Jake Deery from comment #0)
> Hi,
> 
> One of our customers is having an issue with their patron card creator, that
> we have been able to replicate on our support server. Both installations are
> Debian package installations, and both installations run atop Debian 10.
> 
> When one goes to create a 'patron card' using the build-in patron card

Are you using your customer's data? Would it be possible to get a copy of the exact text that is to be put on the card? You can send it directly via email to avoid posting here if needed.

Otherwise if it's sample data, I'll look there.
Comment 8 Liz Rea 2022-08-09 14:27:49 UTC
Hi,

here's an example of what we're trying to do, in field 2 text:

The person whose signature appears on this card accepts responsibility for the appropriate care of all books and materials checked out on this card.

I'll put an attachment of the whole template screen too.

Cheers,
Liz
Comment 9 Liz Rea 2022-08-09 14:28:22 UTC
Created attachment 138917 [details]
screenshot of problem template
Comment 10 Chris Nighswonger 2022-08-09 14:35:11 UTC
(In reply to Liz Rea from comment #8)

Thanks Liz!

To confirm: Removing this text from field two stops the loop from becoming infinite?

> The person whose signature appears on this card accepts responsibility for
> the appropriate care of all books and materials checked out on this card.
Comment 11 Liz Rea 2022-09-12 13:30:56 UTC
It appears to not be infinite when the text doesn't have to wrap.
Comment 12 David Cook 2022-09-12 23:23:58 UTC
(In reply to Liz Rea from comment #11)
> It appears to not be infinite when the text doesn't have to wrap.

Sounds like the issue is in the WRAP_LINES block in C4/Patroncards/Patroncard.pm then (like it was for bug 22429).

I don't have time to look at this today, but I could take a look in the next few days/weeks.
Comment 13 Nick Clemens (kidclamp) 2023-01-23 12:57:01 UTC
Created attachment 145571 [details] [review]
Bug 31259: Escape infinite loop when lime cannot wrap enough

It seems the wrapping here works with words, not characters. If a line is too long,
or too far over, we can't trim it enough. We need to escape if successive loops are
not changing the line

To test:
 1 - Using sample data, edit the layout to match picture in comment 9
 2 - Add some borrowers to a batch
 3 - Attempt to export batch and save PDF
 4 - Staff client waits
 5 - ps aux | grep card
 6 - Note the job keeps running
 7 - Kill it
 8 - Apply patch
 9 - Restart_all
10 - Export batch again
11 - Success, you get a badly formatted PDF, rather than nothing
12 - Note the PDF does not show "Yakama Nation Library"
13 - Edit layout, male "Lower left X" 20 instead of 220
14 - Export again and note text field 1 shows correctly
Comment 14 Chris Nighswonger 2023-01-23 13:15:51 UTC
Comment on attachment 145571 [details] [review]
Bug 31259: Escape infinite loop when lime cannot wrap enough

Review of attachment 145571 [details] [review]:
-----------------------------------------------------------------

I cannot test this, but the logic appears sound, fwiw.
Comment 15 David Nind 2023-01-24 22:16:15 UTC
Created attachment 145639 [details] [review]
Bug 31259: Escape infinite loop when lime cannot wrap enough

It seems the wrapping here works with words, not characters. If a line is too long,
or too far over, we can't trim it enough. We need to escape if successive loops are
not changing the line

To test:
 1 - Using sample data, edit the layout to match picture in comment 9
 2 - Add some borrowers to a batch
 3 - Attempt to export batch and save PDF
 4 - Staff client waits
 5 - ps aux | grep card
 6 - Note the job keeps running
 7 - Kill it
 8 - Apply patch
 9 - Restart_all
10 - Export batch again
11 - Success, you get a badly formatted PDF, rather than nothing
12 - Note the PDF does not show "Yakama Nation Library"
13 - Edit layout, male "Lower left X" 20 instead of 220
14 - Export again and note text field 1 shows correctly

Signed-off-by: David Nind <david@davidnind.com>
Comment 16 David Nind 2023-01-24 22:19:31 UTC
Testing notes (using koha-testing-docker):

1. Tested on the current master.

2. To kill the processes: kill -9 XXXXX (where XXXXX = the 'pid' number, from the second column).

3. I didn't setup the layout to match 100%, but can confirm the issue occurred and the patch fixed things so the PDF was created.
Comment 17 Martin Renvoize (ashimema) 2023-03-15 16:01:31 UTC
Created attachment 148224 [details] [review]
Bug 31259: Escape infinite loop when lime cannot wrap enough

It seems the wrapping here works with words, not characters. If a line is too long,
or too far over, we can't trim it enough. We need to escape if successive loops are
not changing the line

To test:
 1 - Using sample data, edit the layout to match picture in comment 9
 2 - Add some borrowers to a batch
 3 - Attempt to export batch and save PDF
 4 - Staff client waits
 5 - ps aux | grep card
 6 - Note the job keeps running
 7 - Kill it
 8 - Apply patch
 9 - Restart_all
10 - Export batch again
11 - Success, you get a badly formatted PDF, rather than nothing
12 - Note the PDF does not show "Yakama Nation Library"
13 - Edit layout, male "Lower left X" 20 instead of 220
14 - Export again and note text field 1 shows correctly

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 18 Martin Renvoize (ashimema) 2023-03-15 16:04:24 UTC
Nice fix, thanks Nick.

All tested and working here.. QA scripts happy.

I decided to let the lack of Unit test pass here as it's an important fix and the unit tests are already rather bare here.

Passing QA
Comment 19 Tomás Cohen Arazi (tcohen) 2023-03-31 11:13:17 UTC
Pushed to master for 23.05.

Nice work everyone, thanks!
Comment 20 Jacob O'Mara 2023-04-16 17:24:10 UTC
Nice work, thanks everyone!

Pushed to 22.11.x for the next release.
Comment 21 Lucas Gass (lukeg) 2023-04-24 17:31:44 UTC
Backported to 22.05.x for upcoming 22.05.12
Comment 22 Arthur Suzuki 2023-05-03 19:10:32 UTC
applied to 21.11.x for 21.11.20