What are the steps to reproduce
1) In a fresh Koha, generate translation into XX
2) Select XX for the staff interface (i.e. enable it, and choose it)
3) Go to the Patrons page
4) Everything looks fine
5) In the admin page, create a patron category with (say) the non-ASCII character 'ñ'.
6) Go back to the Patrons page, everything is broken

Patch applied cleanly, go forth and signoff

Created attachment 23122 [details] [review]
[PASSED QA] Bug 11290 : Fixing display problem on members-home.pl

To test
1) In a fresh Koha, generate translation into XX
2) Select XX for the staff interface (i.e. enable it, and choose it)
3) Go to the Patrons page
4) Everything looks fine
5) In the admin page, create a patron category with any no-ASCII character.
6) Go back to the Patrons page, everything is broken
7) apply patch
8) regenerate translation
9) go back to the Patrons page, everything looks fine
10) switch to english it still looks fine

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Tried with ru-RU and fixes the problem.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Problem is also visible on untranslated templates changing
one of the sample category descriptions to include diacritics.

Just an additional observation:

This seems to work for me too. I tested a few combinations including Latin diacritics, and some Chinese characters in the patron category as well as in the members template text. The following remark on Apache::Util did however not fully reassure me:

Apache::Util, escape_html says: [This function will correctly escape US-ASCII output. If you are using a different character set such as UTF8, or need more control on the escaping process, use HTML::Entities.]

Note that Template Toolkit (at least version 2.87) does however first try Apache::Util and only if that is not available, HTML::Entities.

Just under the assumption that HTML::Entities would do a better job, we could do the following, forcing TT to use it:
use Template::Filters;
Template::Filters->use_html_entities;
(That would make HTML::Entities a Koha dependency.)

Thank you for the patch, it's working for me as well.

That said, I wonder if instead of adding more and more html_entity to various templates (there is an similar problem with, e.g., bug 10855, and probably more like that), shouldn't we rather fix C4/Templates.pm?
I think the underlying cause for this particular problem in members.pl and members-home.pl is that output()/utf8_arrayref()/utf8_hashref() subs in Templates.pm do not know what to do with objects / blessed hashrefs (they treat them as strings/scalars, actually calling utf8::is_utf8() on them etc.).

(In reply to Jacek Ablewicz from comment #7)
> That said, I wonder if instead of adding more and more html_entity to
> various templates (there is an similar problem with, e.g., bug 10855, and
> probably more like that), shouldn't we rather fix C4/Templates.pm?
> I think the underlying cause for this particular problem in members.pl and
> members-home.pl is that output()/utf8_arrayref()/utf8_hashref() subs in
> Templates.pm do not know what to do with objects / blessed hashrefs (they
> treat them as strings/scalars, actually calling utf8::is_utf8() on them
> etc.).

+1 for a generic solution (if available :)

As an experiment I replaced ref() calls in C4/Templates.pm with reftype() from Scalar::Util package; objects/blessed hashrefs are now processed by output(). But it has an unexepected side effect - in every place html_entity filter is being used (we have 3 such places in koha-tmpl/.../intranet/ ) encoding of filtered part now seems to be broken (removing html_entity filter from templates, or replacing it with html filter fixes it). Or perhaps I'm missing something here, and such side efect is acutally being expected (?).

BTW, html_entity filter is also used in modules/pay.tt - and encoding of this part ([% line.title |html_entity %] ) indeed seems to be broken (same symptoms in unpatched 3.12.4 version too, not just in master/3.15 with patched or unpatched Templates.pm). Does anyone remeber when and way was is added there?

I am also in favor of a generic solution that would help avoid these problems in the future, but I think by pushing this we will get rid of a prominently showing problem quickly. It would give us time to work on the big solution, that will probably require much more time for testing.

(In reply to Jacek Ablewicz from comment #7)
> I think the underlying cause for this particular problem in members.pl and
> members-home.pl is that output()/utf8_arrayref()/utf8_hashref() subs in
> Templates.pm do not know what to do with objects / blessed hashrefs (they
> treat them as strings/scalars, actually calling utf8::is_utf8() on them
> etc.).

Preparing for Reno i had tried even removing any call to those sub. I'd be in favour of removing them and using template filters instead (I.e. less loops traversing data structures and less probable double encoding situations)..

regards
to+

(In reply to Tomás Cohen Arazi from comment #11)
> (In reply to Jacek Ablewicz from comment #7)
> > I think the underlying cause for this particular problem in members.pl and
> > members-home.pl is that output()/utf8_arrayref()/utf8_hashref() subs in
> > Templates.pm do not know what to do with objects / blessed hashrefs (they
> > treat them as strings/scalars, actually calling utf8::is_utf8() on them
> > etc.).
> 
> Preparing for Reno i had tried even removing any call to those sub. I'd be
> in favour of removing them and using template filters instead (I.e. less
> loops traversing data structures and less probable double encoding
> situations)..
> 
I would prefer this situation also

As a stop-gap, I've pushed this to master.  Thanks, Chris!

Pushed to 3.14.x, will be in 3.14.07

Pushed to 3.12.x will be in 3.12.13