Character Encoding

[richmond62]

So: I have 2 sample text files: one in Windows-1251 (Windows encoding for Cyrillic), and one in UTF8 . . .

Code: Select all

on mouseUp
   answer file "A text file"
   if it <> "" then
      put it into theFilePath
      put URL ("file:" & theFilePath) into MYCHEESE
      put textDecode(MYCHEESE, "UTF-8") into fld "f1"
   end if
   put the encoding of fld "f1" into fld "f2"
end mouseUp

Obviously correctly renders the second file . . .

This:

Code: Select all

on mouseUp
   answer file "A text file"
   if it <> "" then
      put it into theFilePath
      put URL ("file:" & theFilePath) into MYCHEESE
      put textDecode(MYCHEESE, "cp1251") into fld "f1"
   end if
   put the encoding of fld "f1" into fld "f2"
end mouseUp

Throws a bluey with the first file (unsurprisingly).

Does anyone know WHERE I can find a list of all the non-unicode character encodings that LiveCode can cope with?

[richmond62]

viewtopic.php?t=23613

http://web.archive.org/web/201503141946 ... ecode-7-0/

“ASCII”

“ISO-8859-1” (Linux only)

“MacRoman” (OSX only)

“Native” (ISO-8859-1 on Linux, MacRoman on OSX, CP1252 Windows)

“UTF-16”

“UTF-16BE”

“UTF-16LE”

“UTF-32”

“UTF-32BE”

“UTF-32LE”

“UTF-8”

“CP1252” (Windows only)

So in no way 'universal'.

[richmond62]

Presumably somewhere "in the undergrowth" of the IDE lurks code for performing these transforms, and presumably other transforms could be added.

[jumoo]

It would be great if LiveCode supported a wider range of legacy encodings directly. Many older text files still use code pages like CP1251, so having built-in support instead of relying on external conversion tools would make handling multilingual data much easier.

[richmond62]

If you are not a Spam bot . . .

It is relatively easy to implement any 'legacy' (and by 'legacy') I take it you mean anything that is not Unicode) coding inwith xTalk.

I have a fantasy that buried somewhere in the xTalk code there is a file/stack that contains a series of lookup tables or somesuch for legacy (i.e. non-Unicode) character encodings . . .

However, I have not yet been able to locate anything like that.