Bug 4472

Summary: img tags in xslt broken after automatic translation
Product: Koha Reporter: Katrin Fischer <katrin.fischer>
Component: I18N/L10NAssignee: Chris Cormack <chris>
Status: CLOSED FIXED QA Contact: Bugs List <koha-bugs>
Severity: major    
Priority: PATCH-Sent (DO NOT USE) CC: cnighswonger, f.demians, fridolin.somers, gmcharlt
Version: rel_3_2   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Proposed patch

Description Chris Cormack 2010-05-21 01:27:35 UTC


---- Reported by katrin.fischer@bsz-bw.de 2010-05-10 11:59:52 ----

I tested Galen's patch to http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=3214 and got an error 500 page, when I tried to search something while German, French or Spanish were chosen.

Searching with English templates worked fine.

With Galen's help we found out that after automatic translation the img tags in MARC21slim2OPACResults.xsl are not closed. This breaks the XSLT files.

I corrected all img tags in my XSLT files and everything worked fine.

Before:

<img src="/opac-tmpl/prog/famfamfam/silk/database.png" alt="Datenbank" title="Datenbank">

After correction:

<img src="/opac-tmpl/prog/famfamfam/silk/database.png" alt="Datenbank" title="Datenbank"/>

MARC21slim2OPACDetail.xsl contains no img tags, but a different solution to create an img tag:

<xsl:element name="img"><xsl:attribute name="src">/opac-tmpl/prog/famfamfam/<xsl:value-of select="$materialTypeCode"/>.png</xsl:attribute><xsl:attribute name="alt"></xsl:attribute></xsl:element>



--- Bug imported by chris@bigballofwax.co.nz 2010-05-21 01:27 UTC  ---

This bug was previously known as _bug_ 4472 at http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=4472

Actual time not defined. Setting to 0.0
Setting qa contact to the default for this product.
   This bug either had no qa contact or an invalid one.

Comment 1 Chris Cormack 2010-06-14 22:02:39 UTC
Patch sent.
Comment 2 Chris Cormack 2010-06-17 08:59:18 UTC
Created attachment 2248 [details] [review]
Proposed patch
Comment 3 Katrin Fischer 2010-06-17 12:36:42 UTC
I applied Chris' patch on my test installation with current master and recreated the German template files. Result and detail pages were displayed correctly.

After that I tried for French and Spanish, but it did not work. 

The img tags in all files look right to me. I will try to retest this later, perhaps someone else can test too.
Comment 4 Galen Charlton 2010-06-17 23:39:24 UTC
Patch pushed - please test and advise if this fixess the problem.
Comment 5 Katrin Fischer 2010-06-18 09:48:55 UTC
I can confirm that the problem with img tags is fixed now.

---------
I investigated a bit more, why my tests with French and Spanish were not successful:

French:

The result list for French does not work, because the xml file can't handle "&nbsp;" in the translated text. The html entities were entered in pootle:

Copies available for reference:
Exemplaires disponibles sur place&nbsp;:

The result page worked after I deleted all nbsp entities.

[Fri Jun 18 10:48:38 2010] [error] [client 193.197.29.224] /home/koha/kohaclone/koha-tmpl/opac-tmpl/prog/fr-FR/xslt/MARC21slim2OPACResults.xsl:973: parser error : Entity 'nbsp' not defined, referer: http://193.197.29.146/cgi-bin/koha/opac-detail.pl?biblionumber=2
[Fri Jun 18 10:48:38 2010] [error] [client 193.197.29.224] Premature end of script headers: opac-search.pl, referer: http://193.197.29.146/cgi-bin/koha/opac-detail.pl?biblionumber=2
[Fri Jun 18 10:48:39 2010] [error] [client 193.197.29.224] [Fri Jun 18 10:48:39 2010] 500.pl: PARAM:LOOP:next_loop:hash pointer was expected but not found at /usr/lib/perl5/HTML/Template/Pro.pm line 191., referer: http://193.197.29.146/cgi-bin/koha/opac-detail.pl?biblionumber=2

---------
Spanish:
It seems that the opac xslt files are not created. The old files with wrong image tags remained. I tried to delete the xslt files and run 'translate install es-ES'. No new files were created.

There is one error for opac files in the process output:
Creating /home/koha/kohaclone/koha-tmpl/opac-tmpl/prog/es-ES/modules/opac-sendbasket.tmpl...
% l: Unknown or unsupported format specification
Charset in/out: UTF-8 at /home/koha/kohaclone/misc/translator/tmpl_process3.pl line 265.
Comment 6 MJ Ray (software.coop) 2010-07-07 19:17:04 UTC
Updating version as described in http://wiki.koha-community.org/wiki/General_IRC_Meeting,_7_July_2010
Comment 7 Frédéric Demians 2010-07-11 15:46:35 UTC
In French any English sentence like 'label:' is translated into 'french
label&nbsp;:' because French typographic rules impose to have a white space
before ':'.

&nbsp; is an entity reference. It comes from PO files and is added in .tmpl and
now in .xsl files. It works with .tmpl files. It doesn't with .xsl files. The
.tmpl tokenizer hasn't been designed to handle properly xml files. 

I see three solutions:

1. It would work if in the .po files &#160; was used rather thant &nbsp; So
   opac and intranet .po files should be modified and reload into Pootle.

2. XSL files are modified: an XML entity is added for nbsp. tmpl_process3.pl is
   modified to handle specifically <DOCTYPE> and <ENTITY> directives.

3. XSL files are tokenized independently than .tmpl files. This is the best
   solution for the long run.
Comment 8 Frédéric Demians 2010-07-11 15:54:36 UTC
Solution 2:

http://lists.koha-community.org/pipermail/koha-patches/2010-July/012259.html
Comment 9 Fridolin Somers 2010-09-24 12:22:33 UTC
(In reply to comment #8)
> Solution 2:
> 
> http://lists.koha-community.org/pipermail/koha-patches/2010-July/012259.html

Why not modify the po files with &#160; where xslt occures ?
Comment 10 Fridolin Somers 2010-09-24 12:24:39 UTC
(In reply to comment #8)
> Solution 2:
> 
> http://lists.koha-community.org/pipermail/koha-patches/2010-July/012259.html

Why not modify the po files with &#160; where xslt occures ?(In reply to comment #9)
> (In reply to comment #8)
> > Solution 2:
> > 
> > http://lists.koha-community.org/pipermail/koha-patches/2010-July/012259.html
> 
> Why not modify the po files with &#160; where xslt occures ?

More efficient : define a different po file for xslt files.
Comment 11 Frédéric Demians 2010-09-24 13:08:54 UTC
> Why not modify the po files with &#160; where xslt occures ?

It could be a solution but I'm almost sure that &nbsp; are added automatically and blindly by whichever .po file editor is used by translators.

> More efficient : define a different po file for xslt files.

I agree. That's my point 3...

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=4472#c7

It implies developing a new XSL file tokenizer and a new way to insert translated string into XSL files (translate install).
Comment 12 Frédéric Demians 2010-10-17 14:24:31 UTC
This patch fixes this bug in Intranet detail page, for UNIMARC DB:

http://lists.koha-community.org/pipermail/koha-patches/2010-October/012687.html
Comment 13 Galen Charlton 2010-10-17 17:31:49 UTC
(In reply to comment #12)
> This patch fixes this bug in Intranet detail page, for UNIMARC DB:
> 
> http://lists.koha-community.org/pipermail/koha-patches/2010-October/012687.html

Patch pushed to HEAD for inclusion in 3.2.
Comment 14 Galen Charlton 2010-10-17 17:33:36 UTC
Changing bug priority and moving to 3.4, now that remaining nbsp fix is in place.
Comment 15 Chris Cormack 2010-11-08 03:26:52 UTC
Patch for MARC21 pushed also, please test and close
Comment 16 MJ Ray (software.coop) 2010-11-09 04:18:07 UTC
This bug is mentioned in:
Bug 4472 Fix how &amp;nbsp; are injected into XSL files http://lists.koha-community.org/pipermail/koha-patches/2010-October/012640.html
Comment 17 Chris Nighswonger 2010-11-18 03:15:34 UTC
This bug has a patch presently committed to HEAD and 3.2.x.

Please take the appropriate action to update the status of this bug.

Remember, bugs which have been resolved should be tested and marked closed, preferably by the original reporter.
Comment 18 Chris Cormack 2010-11-20 05:18:11 UTC
This is still waiting on testing.
Comment 19 Katrin Fischer 2010-11-23 11:35:30 UTC
Tested in 3.2.1: Intranet detail view with MARC21 in French is still broken. OPAC works fine. Follow up patch not pushed?
Comment 20 Chris Nighswonger 2010-11-30 13:49:38 UTC
This bug has a fix which was committed to the 3.2.x branch and released in 3.2.1.

Please take time to update the status of this bug and mark it RESOLVED FIXED so that it may be closed.
Comment 21 MJ Ray (software.coop) 2010-12-09 15:10:28 UTC
This bug is mentioned in:
Bug 4472 Apply it to MARC21 http://lists.koha-community.org/pipermail/koha-patches/2010-November/012831.html
Comment 22 Katrin Fischer 2010-12-13 19:37:39 UTC
Retested in current 3.2.1 and intranet detail view for MARC21 in French is still broken. OPAC works fine. Follow up patch not pushed?
Comment 23 Katrin Fischer 2011-01-10 17:40:36 UTC
I think this patch is still missing in 3.2 - causing the problems with French/MARC21:

http://git.koha-community.org/gitweb/?p=koha.git;a=commit;h=c616cb5081b4e96a586fae1e5601d5d495c8794d
Comment 24 Chris Nighswonger 2011-01-10 18:49:36 UTC
(In reply to comment #23)
> I think this patch is still missing in 3.2 - causing the problems with
> French/MARC21:
> 
> http://git.koha-community.org/gitweb/?p=koha.git;a=commit;h=c616cb5081b4e96a586fae1e5601d5d495c8794d

See the following 3.2.x commit:

http://git.koha-community.org/gitweb/?p=koha.git;a=commit;h=b326530722895a1fd7fa4e0a97e001930ac49083
Comment 25 Katrin Fischer 2011-01-10 18:54:52 UTC
I will retest... perhaps the logs or something else will tell me what the problem is.
Comment 26 Katrin Fischer 2011-03-06 20:45:16 UTC
Successfully tested fr-FR, XSLT ON, MARC21 on master and 3.2.x