Bug 31883

Summary: Filter trim causes false warnings
Product: Koha Reporter: Mason James <mtj>
Component: Test SuiteAssignee: Mason James <mtj>
Status: RESOLVED FIXED QA Contact: Testopia <testopia>
Severity: normal    
Priority: P1 - high CC: arthur.suzuki, david, lucas, m.de.rooy, martin.renvoize, tomascohen, wainuiwitikapark
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
22.11.00, 22.05.08, 21.11.15
Circulation function:
Attachments: Bug 31883: t::lib::QA::TemplateFilters missing built-in TT filter "trim" causes false warnings
Filter trim causes false warnings
Bug 31883: t::lib::QA::TemplateFilters missing built-in TT filter "trim" causes false warnings
Bug 31883: Filter trim causes false warnings
Bug 31883: Filter trim causes false warnings

Description Mason James 2022-10-20 00:27:00 UTC
info coming...
Comment 1 Mason James 2022-10-20 00:33:26 UTC
Created attachment 142218 [details] [review]
Bug 31883: t::lib::QA::TemplateFilters missing built-in TT filter "trim" causes false warnings

to test...

    1/ checkout latest master

    2/ prove xt/find-missing-filters.t
    xt/find-missing-filters.t .. 1/1
    #   Failed test 'Template variables should be correctly escaped'
    #   at xt/find-missing-filters.t line 59.
    #          got: '1'
    #     expected: '0'
    # $VAR1 = {
    #           'errors' => [
    #                         {
    #                           'error' => 'missing_filter',
    #                           'line' => '                        [% messages = INCLUDE \'patron_messages.inc\' moremember = 1 | trim %]',
    #                           'line_number' => 97
    #                         }
    #                       ],
    #           'file' => 'koha-tmpl/intranet-tmpl/prog/en/modules/members/moremember.tt'
    #         };
    # Looks like you failed 1 test of 1.

    3/ apply patch

    4/ prove xt/find-missing-filters.t

    xt/find-missing-filters.t .. ok
    All tests successful.
    Files=1, Tests=1,  2 wallclock secs ( 0.03 usr  0.01 sys +  2.15 cusr  0.07 csys =  2.26 CPU)
    Result: PASS <<<<<<<<<<<<<<<
Comment 2 Tomás Cohen Arazi (tcohen) 2022-10-20 01:05:53 UTC
Mason, thanks for the patch! I thought of a similar patch, but kept thinking a variable assignment shouldn't be forced to have a filter, and so was in doubt about it. I'm happy to go with this regardless.
Comment 3 David Nind 2022-10-20 07:24:06 UTC
I followed the test plan and everything works. For some reason I can't sign off like I normally do (git bz attach -e 31883 HEAD).
Comment 4 David Nind 2022-10-20 07:35:22 UTC
I can sign off other bugs, so maybe something wrong with this one?

This is the message I'm getting when I go git bz attach -e 31883 HEAD (after a git so 1):

git bz attach -e 31883 HEAD
/usr/bin/git-bz:1908: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  obsoleted = (initial_description == patch.description)
Traceback (most recent call last):
  File "/usr/bin/git-bz", line 2722, in <module>
    do_attach(*args)
  File "/usr/bin/git-bz", line 2074, in do_attach
    attach_commits(bug, commits, edit_comments=global_options.edit)
  File "/usr/bin/git-bz", line 1988, in attach_commits
    description, body, obsoletes, statuses, patch_complexities, depends = edit_attachment_comment(bug, commit.subject, body)
  File "/usr/bin/git-bz", line 1940, in edit_attachment_comment
    lines = edit_template(template.getvalue())
  File "/usr/lib/python2.7/StringIO.py", line 271, in getvalue
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3: ordinal not in range(128)
Comment 5 Marcel de Rooy 2022-10-20 07:55:00 UTC
Same here

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3: ordinal not in range(128)

Will edit commit message now.
Comment 6 Marcel de Rooy 2022-10-20 07:55:48 UTC
Lowering severity. Critical is really another category :)
Comment 7 Marcel de Rooy 2022-10-20 08:11:48 UTC
Not sure whats going on here.
Please resubmit a new version.
Comment 8 Marcel de Rooy 2022-10-20 08:14:21 UTC
It seems to be another general issue?
Comment 9 Marcel de Rooy 2022-10-20 09:06:55 UTC
(In reply to Marcel de Rooy from comment #8)
> It seems to be another general issue?

No. I refreshed my container. Could git attach again.
But after applying this patch, we see our friend again:

/usr/local/bin/git-bz:1908: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  obsoleted = (initial_description == patch.description)
Traceback (most recent call last):
  File "/usr/local/bin/git-bz", line 2722, in <module>
    do_attach(*args)
  File "/usr/local/bin/git-bz", line 2074, in do_attach
    attach_commits(bug, commits, edit_comments=global_options.edit)
  File "/usr/local/bin/git-bz", line 1988, in attach_commits
    description, body, obsoletes, statuses, patch_complexities, depends = edit_attachment_comment(bug, commit.subject, body)
  File "/usr/local/bin/git-bz", line 1940, in edit_attachment_comment
    lines = edit_template(template.getvalue())
  File "/usr/lib/python2.7/StringIO.py", line 271, in getvalue
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3: ordinal not in range(128)
Comment 10 Marcel de Rooy 2022-10-20 09:19:12 UTC
Comment on attachment 142218 [details] [review]
Bug 31883: t::lib::QA::TemplateFilters missing built-in TT filter "trim" causes false warnings

Ttile of bug report and attachment name cause(d) Unicode issues
Comment 11 Marcel de Rooy 2022-10-20 09:19:29 UTC
Created attachment 142235 [details] [review]
Filter trim causes false warnings
Comment 12 Martin Renvoize (ashimema) 2022-10-20 09:19:39 UTC Comment hidden (obsolete)
Comment 13 Marcel de Rooy 2022-10-20 09:25:08 UTC
Created attachment 142237 [details] [review]
Bug 31883: Filter trim causes false warnings

Test plan:
Run xt/find-missing-filters.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 14 Martin Renvoize (ashimema) 2022-10-20 09:27:19 UTC
Created attachment 142238 [details] [review]
Bug 31883: Filter trim causes false warnings

Test plan:
Run xt/find-missing-filters.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 15 Tomás Cohen Arazi (tcohen) 2022-10-20 14:44:58 UTC
Pushed to master for 22.11.

Nice work everyone, thanks!
Comment 16 Lucas Gass (lukeg) 2022-12-05 20:50:24 UTC
Backported to 22.05.x for upcoming 22.05.08
Comment 17 Arthur Suzuki 2022-12-16 13:31:20 UTC
applied to 21.11 for 21.11.15
Comment 18 Wainui Witika-Park 2023-03-15 00:57:39 UTC
Not backported to 21.05.x