Bug 23777 - Text converted to html entity codes in cataloguing edit form
Summary: Text converted to html entity codes in cataloguing edit form
Status: Pushed to master
Alias: None
Product: Koha
Classification: Unclassified
Component: Cataloging (show other bugs)
Version: master
Hardware: All Linux
: P4 normal (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Keywords:
Depends on: 13618
Blocks:
  Show dependency treegraph
 
Reported: 2019-10-08 13:50 UTC by Bernard
Modified: 2020-03-13 01:22 UTC (History)
9 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:
20.05.00


Attachments
Bug 23777: Fix auth finder escaping (1.73 KB, patch)
2019-11-11 18:04 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Fix for the advanced editor (1.92 KB, patch)
2019-11-12 08:19 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Use To.json in preferences.tt (1.37 KB, patch)
2019-11-12 08:37 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Use To.json in authorities.tt (1.62 KB, patch)
2019-11-12 08:37 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Use To.json in results.tt (1.36 KB, patch)
2019-11-12 08:37 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Use TO.json in value_builder/unimarc_field_4XX.tt (6.74 KB, patch)
2019-11-12 08:37 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Use To.json in request.tt (2.08 KB, patch)
2019-11-12 08:37 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Fix auth finder escaping (1.79 KB, patch)
2020-01-22 03:35 UTC, Hayley Mapley
Details | Diff | Splinter Review
Bug 23777: Fix for the advanced editor (1.98 KB, patch)
2020-01-22 03:35 UTC, Hayley Mapley
Details | Diff | Splinter Review
Bug 23777: Use To.json in preferences.tt (1.43 KB, patch)
2020-01-22 03:35 UTC, Hayley Mapley
Details | Diff | Splinter Review
Bug 23777: Use To.json in authorities.tt (1.68 KB, patch)
2020-01-22 03:36 UTC, Hayley Mapley
Details | Diff | Splinter Review
Bug 23777: Use To.json in results.tt (1.42 KB, patch)
2020-01-22 03:37 UTC, Hayley Mapley
Details | Diff | Splinter Review
Bug 23777: Use TO.json in value_builder/unimarc_field_4XX.tt (6.80 KB, patch)
2020-01-22 03:39 UTC, Hayley Mapley
Details | Diff | Splinter Review
Bug 23777: Use To.json in request.tt (2.14 KB, patch)
2020-01-22 03:40 UTC, Hayley Mapley
Details | Diff | Splinter Review
Bug 23777: Use To.json in results.tt (1.33 KB, patch)
2020-02-11 10:25 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Use TO.json in value_builder/unimarc_field_4XX.tt (6.80 KB, patch)
2020-02-11 10:25 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Use To.json in request.tt (2.14 KB, patch)
2020-02-11 10:25 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 23777: Fix auth finder escaping (1.85 KB, patch)
2020-02-21 07:01 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 23777: Fix for the advanced editor (2.04 KB, patch)
2020-02-21 07:01 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 23777: Use To.json in preferences.tt (1.49 KB, patch)
2020-02-21 07:01 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 23777: Use To.json in authorities.tt (1.74 KB, patch)
2020-02-21 07:01 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 23777: Use To.json in results.tt (1.39 KB, patch)
2020-02-21 07:01 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 23777: Use TO.json in value_builder/unimarc_field_4XX.tt (6.86 KB, patch)
2020-02-21 07:01 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 23777: Use To.json in request.tt (2.20 KB, patch)
2020-02-21 07:01 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 23777: (QA follow-up) Fix TT closing braces (5.29 KB, patch)
2020-02-21 07:04 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 23777: (follow-up) Use To.json in results.tt (1.03 KB, patch)
2020-02-25 11:01 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Bernard 2019-10-08 13:50:25 UTC
1. Create corporate authority record which has double-quotes: eg Fundación "La Caixa" (Madrid, Spain)
2. Edit catalogue record where 710 field under authority control.
3. Click gadget in pipe a and searcg for authority
4. Click Choose
5. Observe the catalogue record 710 field now says Fundación "La Caixa" (Madrid, Spain)
The entity codes persist when record is saved and viewed elsewhere in the system.
Comment 1 Bernard 2019-10-08 13:52:42 UTC
Temporary fix in /kohaclone/koha-tmpl/intranet-tmpl/prog/en-GB/modules/authorities/blinddetail-biblio-search.tt is to add 

if(values[i].indexOf('"')) {
   values[i] = values[i].replace(/\&quot\;/g, '"');
}

after line 99, but only deals with " entity, not others.
Comment 2 Theodoros Theodoropoulos 2019-10-21 07:20:36 UTC
I have the exact same issue.
Although it seems that the issue affects only the *display* of the value that is under authority control (and not the value itself), I believe that the bug should have a higher priority. If will be intimidating for cataloguers to see """ in authority controlled fields.
Comment 3 Jonathan Druart 2019-11-11 18:04:31 UTC
Created attachment 95275 [details] [review]
Bug 23777: Fix auth finder escaping
Comment 4 Jonathan Druart 2019-11-11 18:05:19 UTC
I will need to double check this fix, but it seems ok at first glance.
Comment 5 Jonathan Druart 2019-11-12 08:19:48 UTC
Created attachment 95279 [details] [review]
Bug 23777: Fix for the advanced editor
Comment 6 Jonathan Druart 2019-11-12 08:37:21 UTC
Created attachment 95280 [details] [review]
Bug 23777: Use To.json in preferences.tt

Test plan:
Search for "foo bar" (with double quotes) in the sysprefs
Comment 7 Jonathan Druart 2019-11-12 08:37:25 UTC
Created attachment 95281 [details] [review]
Bug 23777: Use To.json in authorities.tt

No fix here.

Test plan:
Edit GENRE/FORM 040$a
With text: Original catalo'"ging age"'ncy
Save, you get the alert
Comment 8 Jonathan Druart 2019-11-12 08:37:29 UTC
Created attachment 95282 [details] [review]
Bug 23777: Use To.json in results.tt

Test plan:
Search for a string with quote and confirm that it is correctly
highlighted in the result list.
Comment 9 Jonathan Druart 2019-11-12 08:37:32 UTC
Created attachment 95283 [details] [review]
Bug 23777: Use TO.json in value_builder/unimarc_field_4XX.tt

No idea how to test this but, as other fixes work, this one should,
right?
Comment 10 Jonathan Druart 2019-11-12 08:37:36 UTC
Created attachment 95284 [details] [review]
Bug 23777: Use To.json in request.tt

Test plan:
Use quotes in a library's name, then place an item on hold for a patron
that cannot do it.
If you are allowed to overwrite this restriction you will get a warning:
"This item normally cannot be put on hold except for patrons from
BRANCHNAME"
The library's name should be correcty displayed
Comment 11 Jonathan Druart 2019-11-12 08:38:04 UTC
Last 5 patches fix similar issues in other templates.
Comment 12 Bernard 2019-11-12 10:40:36 UTC
Thanks for looking at this Jonathan.

Using same workflow as in first post (2019-10-08 13:50:25 UTC):

patch 95275 causes $9 field to be completed (link) but $a field to remain blank after clicking "choose". The entire field is discarded on "save".

Adding patch 95279 - ditto but there is now no $9 field either.
Comment 13 Jonathan Druart 2019-11-12 11:01:53 UTC
What I see: https://snipboard.io/zuKCU0.jpg

Everything looks correct for me, can you provide more details? Are you testing on a sandbox?
Comment 14 Bernard 2019-11-12 11:34:21 UTC
I'm on the customer test server where we had the original issue. 18.11.04.
Comment 15 Jonathan Druart 2019-11-12 11:51:52 UTC
(In reply to Bernard from comment #14)
> I'm on the customer test server where we had the original issue. 18.11.04.

Tested locally with this version + the patches, all good.
Do you see an error in the console?
Comment 16 Bernard 2019-11-12 14:25:30 UTC
Nothing in the js console. When you click "choose", the small pop up window appears briefly saying "loading", but then Koha just returns to the main catalogue record window.
Comment 17 Hayley Mapley 2020-01-22 03:35:00 UTC
Created attachment 97685 [details] [review]
Bug 23777: Fix auth finder escaping

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 18 Hayley Mapley 2020-01-22 03:35:31 UTC
Created attachment 97686 [details] [review]
Bug 23777: Fix for the advanced editor

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 19 Hayley Mapley 2020-01-22 03:35:59 UTC
Created attachment 97687 [details] [review]
Bug 23777: Use To.json in preferences.tt

Test plan:
Search for "foo bar" (with double quotes) in the sysprefs

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 20 Hayley Mapley 2020-01-22 03:36:46 UTC
Created attachment 97688 [details] [review]
Bug 23777: Use To.json in authorities.tt

No fix here.

Test plan:
Edit GENRE/FORM 040$a
With text: Original catalo'"ging age"'ncy
Save, you get the alert

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 21 Hayley Mapley 2020-01-22 03:37:57 UTC
Created attachment 97689 [details] [review]
Bug 23777: Use To.json in results.tt

Test plan:
Search for a string with quote and confirm that it is correctly
highlighted in the result list.

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 22 Hayley Mapley 2020-01-22 03:39:46 UTC
Created attachment 97690 [details] [review]
Bug 23777: Use TO.json in value_builder/unimarc_field_4XX.tt

No idea how to test this but, as other fixes work, this one should,
right?

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 23 Hayley Mapley 2020-01-22 03:40:43 UTC
Created attachment 97691 [details] [review]
Bug 23777: Use To.json in request.tt

Test plan:
Use quotes in a library's name, then place an item on hold for a patron
that cannot do it.
If you are allowed to overwrite this restriction you will get a warning:
"This item normally cannot be put on hold except for patrons from
BRANCHNAME"
The library's name should be correcty displayed

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 24 Bouzid Fergani 2020-02-07 19:35:26 UTC
Hi,
Thanks to rebase this patch.
Bug 23777 - Text converted to html entity codes in cataloguing edit form

97685 - Bug 23777: Fix auth finder escaping
97686 - Bug 23777: Fix for the advanced editor
97687 - Bug 23777: Use To.json in preferences.tt
97688 - Bug 23777: Use To.json in authorities.tt
97689 - Bug 23777: Use To.json in results.tt
97690 - Bug 23777: Use TO.json in value_builder/unimarc_field_4XX.tt
97691 - Bug 23777: Use To.json in request.tt

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 23777: Fix auth finder escaping
Applying: Bug 23777: Fix for the advanced editor
Applying: Bug 23777: Use To.json in preferences.tt
Applying: Bug 23777: Use To.json in authorities.tt
Applying: Bug 23777: Use To.json in results.tt
Using index info to reconstruct a base tree...
M	koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/results.tt
Falling back to patching base and 3-way merge...
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/results.tt
CONFLICT (content): Merge conflict in koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/results.tt
error: Failed to merge in the changes.
Patch failed at 0001 Bug 23777: Use To.json in results.tt
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-23777-Use-Tojson-in-resultstt-VwtpJU.patch

Thanks.
Bouzid.
Comment 25 Jonathan Druart 2020-02-11 10:25:06 UTC
Created attachment 98698 [details] [review]
Bug 23777: Use To.json in results.tt

Test plan:
Search for a string with quote and confirm that it is correctly
highlighted in the result list.

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 26 Jonathan Druart 2020-02-11 10:25:10 UTC
Created attachment 98699 [details] [review]
Bug 23777: Use TO.json in value_builder/unimarc_field_4XX.tt

No idea how to test this but, as other fixes work, this one should,
right?

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 27 Jonathan Druart 2020-02-11 10:25:14 UTC
Created attachment 98700 [details] [review]
Bug 23777: Use To.json in request.tt

Test plan:
Use quotes in a library's name, then place an item on hold for a patron
that cannot do it.
If you are allowed to overwrite this restriction you will get a warning:
"This item normally cannot be put on hold except for patrons from
BRANCHNAME"
The library's name should be correcty displayed

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 28 Martin Renvoize 2020-02-21 07:01:26 UTC
Created attachment 99341 [details] [review]
Bug 23777: Fix auth finder escaping

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 29 Martin Renvoize 2020-02-21 07:01:29 UTC
Created attachment 99342 [details] [review]
Bug 23777: Fix for the advanced editor

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 30 Martin Renvoize 2020-02-21 07:01:33 UTC
Created attachment 99343 [details] [review]
Bug 23777: Use To.json in preferences.tt

Test plan:
Search for "foo bar" (with double quotes) in the sysprefs

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 31 Martin Renvoize 2020-02-21 07:01:36 UTC
Created attachment 99344 [details] [review]
Bug 23777: Use To.json in authorities.tt

No fix here.

Test plan:
Edit GENRE/FORM 040$a
With text: Original catalo'"ging age"'ncy
Save, you get the alert

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 32 Martin Renvoize 2020-02-21 07:01:39 UTC
Created attachment 99345 [details] [review]
Bug 23777: Use To.json in results.tt

Test plan:
Search for a string with quote and confirm that it is correctly
highlighted in the result list.

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 33 Martin Renvoize 2020-02-21 07:01:43 UTC
Created attachment 99346 [details] [review]
Bug 23777: Use TO.json in value_builder/unimarc_field_4XX.tt

No idea how to test this but, as other fixes work, this one should,
right?

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 34 Martin Renvoize 2020-02-21 07:01:46 UTC
Created attachment 99347 [details] [review]
Bug 23777: Use To.json in request.tt

Test plan:
Use quotes in a library's name, then place an item on hold for a patron
that cannot do it.
If you are allowed to overwrite this restriction you will get a warning:
"This item normally cannot be put on hold except for patrons from
BRANCHNAME"
The library's name should be correcty displayed

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 35 Martin Renvoize 2020-02-21 07:04:26 UTC
Created attachment 99348 [details] [review]
Bug 23777: (QA follow-up) Fix TT closing braces

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 36 Martin Renvoize 2020-02-21 07:05:34 UTC
Great to see these moving to more traditional filters/escaping/parsing practices.

QA scripts are happy and in testing I found all cases working.

Passing QA
Comment 37 Martin Renvoize 2020-02-21 15:45:13 UTC
Nice work everyone!

Pushed to master for 20.05
Comment 38 Jonathan Druart 2020-02-25 11:01:23 UTC
Created attachment 99570 [details] [review]
Bug 23777: (follow-up) Use To.json in results.tt

Wrong copy/paste here!
Comment 39 Martin Renvoize 2020-02-25 11:07:31 UTC
Nice work everyone!

Pushed to master for 20.05
Comment 40 Owen Leonard 2020-02-26 12:31:24 UTC
*** Bug 24713 has been marked as a duplicate of this bug. ***
Comment 41 Joy Nelson 2020-03-13 01:22:37 UTC
does not apply to 19.11.x branch cleanly.
please rebase if needed