Bug 8963 - Can not upload csv file for Quote of the day and encoding problems with non-UTF-8 files
Summary: Can not upload csv file for Quote of the day and encoding problems with non-U...
Status: CLOSED WONTFIX
Alias: None
Product: Koha
Classification: Unclassified
Component: Staff interface (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Bugs List
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-10-24 18:56 UTC by Marc Véron
Modified: 2016-01-28 20:03 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marc Véron 2012-10-24 18:56:04 UTC
Steps to reproduce:

Go to Home › Tools › Quote editor › Quote uploader

Choose a .csv file to upload for the "Quotes of the Day"

You get an alert: "Uploads limited to csv. Incorrect filetype: unknown" and the file is cleared.

It seems that in koha-tmpl/intranet-tmpl/prog/en/modules/tools/quotes-upload.tt, JavaScript function fnHandleFileSelect(evt), the value of evt.target.files[0].type is empty or undefined.

Other properties of evt.target.files[0] like file name or size are populated as expected, and with .txt files the type resolves to txt/plain.

Tested with Firefox 16.01 and Chrome  22.0.1229.94 m on Windows 7

Workaround: Do not use uploader, insert quotes manually.
Comment 1 Marc Véron 2012-11-07 16:37:58 UTC
Update:

I did a test by changing the code after "sanity check" in koha-tmpl/intranet-tmpl/prog/en/modules/toolsto accept .txt files as well.

While this would provide a workaround, I got an other glitch: 

If the file is not UTF-8 encoded, you get garbled characters with umlauts etc.

This is due to following line (no encoding paramter given):

reader.readAsText(evt.target.files[0]);

FileReader.readAsText(File, encoding) defaults to encoding = "UTF-8".

It seems to be difficult to determine the file's encoding for use param, see:
http://stackoverflow.com/questions/10338235/javascript-reading-local-text-file-charset-issue

--snip----
You'd need to analyze the raw binaries of the text file to have a best guess at what the encoding is. There isn't any libraries for this in javascript AFAIK but you could port one from other languages.

Since that isn't very robust, you should also provide a manual override like Characters not showing correctly? Change encoding:
--snip----