[solved] What happened to native encoding in LC8?

[Ledigimate]

Hi livecoders,

When I type normal characters in an input field in LiveCode 8, e.g. abc123, the encoding of each character is reported as "unicode". Isn't it supposed be be "native"? I don't know if I'm just being silly or what, but It would seem like native encoding is no longer supported in LiveCode 8.

Can anyone confirm this?

Regards,

Gerrie

[jacque]

Native encoding refers to the encoding used by the current OS, which varies by platform. LC uses UTF16 in its fields, which is unicode.

[Ledigimate]

Thank you jacque;

Previous versions of LC used to treat the ordinary typed characters found on a standard US keyboard as "native". If this behavior is no longer supported in LC8, it breaks my htmlText formatting handlers which examine the encoding of each character to determine how its html equivalent should be rendered.

How, if possible, should I go about enabling my fields to once again treat the ordinary typed characters as "native", since the useUnicode property has been depricated?

[peter-b]

> Previous versions of LC used to treat the ordinary typed characters found on a standard US keyboard as "native". If this behavior is no longer supported in LC8, it breaks my htmlText formatting handlers which examine the encoding of each character to determine how its html equivalent should be rendered.

I think you should post the affected "htmlText formatting handler" here so that we can see what you mean. Off the top of my head, I don't see why this should be an issue. I think that it might actually be quite a lot easier to implement the handler you describe "correctly" using LiveCode 7+ Unicode support.

[jacque]

Yes, please do post your script. But if I understand it right, I don't think you will need your handler any more. Since LC 7, all text in a field is unicode. There's nothing else. The html text should be correct automatically.

Before LC7, field text was a mixture of unicode and non-unicode characters. That's no longer true, it's UTF16 all the time now.

[Ledigimate]

Try the folllowing in LC versions prior to LC7:

Create a new stack and put the following script into the card script:

Code: Select all

global superScriptArray, superScriptArrayTrans

on openCard
   if superScriptArray is not an array then -- superScriptArray has not been initialized yet
      put baseConvert("2070",16,10) into superScriptArray["0"]
      put baseConvert("00B9",16,10) into superScriptArray["1"]
      put baseConvert("00B2",16,10) into superScriptArray["2"]
      put baseConvert("00B3",16,10) into superScriptArray["3"]
      put baseConvert("2074",16,10) into superScriptArray["4"]
      put baseConvert("2075",16,10) into superScriptArray["5"]
      put baseConvert("2076",16,10) into superScriptArray["6"]
      put baseConvert("2077",16,10) into superScriptArray["7"]
      put baseConvert("2078",16,10) into superScriptArray["8"]
      put baseConvert("2079",16,10) into superScriptArray["9"]
      
      put baseConvert("1D43",16,10) into superScriptArray["a"]
      put baseConvert("1D47",16,10) into superScriptArray["b"]
      put baseConvert("1D9C",16,10) into superScriptArray["c"]
      put baseConvert("1D48",16,10) into superScriptArray["d"]
      put baseConvert("1D49",16,10) into superScriptArray["e"]
      put baseConvert("1DA0",16,10) into superScriptArray["f"]
      put baseConvert("1DA2",16,10) into superScriptArray["g"]
      put baseConvert("02B0",16,10) into superScriptArray["h"]
      put baseConvert("2071",16,10) into superScriptArray["i"]
      put baseConvert("02B2",16,10) into superScriptArray["j"]
      put baseConvert("1D4F",16,10) into superScriptArray["k"]
      put baseConvert("02E1",16,10) into superScriptArray["l"]
      put baseConvert("1D50",16,10) into superScriptArray["m"]
      put baseConvert("207F",16,10) into superScriptArray["n"]
      put baseConvert("1D52",16,10) into superScriptArray["o"]
      put baseConvert("1D56",16,10) into superScriptArray["p"]
      -- no superScriptArray available for q
      put baseConvert("02B3",16,10) into superScriptArray["r"]
      put baseConvert("02E2",16,10) into superScriptArray["s"]
      put baseConvert("1D57",16,10) into superScriptArray["t"]
      put baseConvert("1D58",16,10) into superScriptArray["u"]
      put baseConvert("1D5B",16,10) into superScriptArray["v"]
      put baseConvert("02B7",16,10) into superScriptArray["w"]
      put baseConvert("02E3",16,10) into superScriptArray["x"]
      put baseConvert("02B8",16,10) into superScriptArray["y"]
      put baseConvert("1DBB",16,10) into superScriptArray["z"]
      
      put baseConvert("207A",16,10) into superScriptArray["+"]
      put baseConvert("207B",16,10) into superScriptArray["-"]
      put baseConvert("207C",16,10) into superScriptArray["="]
      put baseConvert("207D",16,10) into superScriptArray["("]
      put baseConvert("207E",16,10) into superScriptArray[")"]
      
      repeat for each char tChar in the keys of superScriptArray
         put tChar into superScriptArrayTrans[superScriptArray[tChar]]
      end repeat
   end if
end openCard

function htmlLiteral theChar
   put theChar into field fldHtmlEntity
   return char 4 to -5 of the htmlText of field fldHtmlEntity -- parse the html entity of the given character
end htmlLiteral

function formattedInput2 theFieldID, theExponents
   -- returns htmlText for the specified field, formatting the characters in the specified positions as exponents, i.e. superscript.
   -- theFieldID:  the short id of the field
   -- theExponents:  a list of the positions of the characters that should be converted to exponents, i.e. superscript.
   -- there MUST be an invisible field called fldHtmlEntity on every card that calls this function!
   local thehtmltext /*the function's return value*/, htt /*an htmlText chunk*/
   repeat with i = 1 to the length of field id theFieldID
      if the encoding of char i of field id theFieldID is native then -- character i is NATIVE, i.e. not unicode
         if i is among the items of theExponents then -- character i should be changed to an exponent, i.e. superscript
            put superScriptArray[char i of field id theFieldID] into tCode -- superScriptArray's elements contain the unicode number of its keys' superscript.
            if tCode is not empty then -- a superscript equivalent for character i exists
               if tCode <= 255 then -- not all superscript characters are unicode; some are ASCII (native)
                  set the useUnicode to false
                  put numToChar(tCode) into field fldHtmlEntity
               else -- the superscript character is UNICODE
                  set the useUnicode to true
                  put unicode numToChar(tCode) into field fldHtmlEntity
               end if
               put char 4 to -5 of the htmlText of field fldHtmlEntity into htt -- parse the html entity of the superscript character
            else -- character i has no superscript equivalent, so leave it as it is
               put char i of field id theFieldID into tChar
               put htmlLiteral(tChar) into htt
            end if
         else -- the character should be normal (NOT an exponent), i.e. not superscript
            -- Some superscipt characters are native, so we need to check for them in order to change them to normal characters too.
            set the useUnicode to false
            put charToNum(char i of field id theFieldID) into tNum
            put superScriptArrayTrans[tNum] into tChar -- superScriptArrayTrans is the inverse of superScriptArray.
            --                                                                  Its elements contain the character representation of its numeric keys.
            if tChar is not empty then
               if tChar is among the chars of "abcdefghijklmnopqrstuvwxyz" then
                  put "<i>" & tChar & "</i>" into htt
               else
                  put htmlLiteral(tChar) into htt
               end if
            else
               if char i of field id theFieldID is among the chars of "abcdefghijklmnopqrstuvwxyz" then
                  put "<i>" & char i of field id theFieldID & "</i>" into htt
               else
                  put htmlLiteral(char i of field id theFieldID) into htt
               end if
            end if
         end if
         put htt after thehtmltext
      else -- character i is UNICODE, which implies that it is a superscript character
         put unicode char (i*2) -1 to (i*2) of the unicodeText of field id theFieldID into field fldHtmlEntity
         if i is among the items of theExponents then -- character i should be an exponent, but it is already a superscript character
            put char 4 to -5 of the htmlText of field fldHtmlEntity into htt -- parse the html entity of the superscript character
         else -- character i should be normal (NOT an exponment), i.e. not superscipt
            set the useUnicode to true
            put unicode charToNum(the text of field fldHtmlEntity) into tNum
            put superScriptArrayTrans[tNum] into tChar -- superScriptArrayTrans is the inverse of superScriptArray.  Its elements contain the character representation of its numeric keys.
            if tChar is not empty then
               if tChar is among the chars of "abcdefghijklmnopqrstuvwxyz" then
                  put "<i>" & tChar & "</i>" into htt
               else
                  put htmlLiteral(tChar) into htt
               end if
            else
               -- we don't expect to encounter unicode characters aren't in superScriptArrayTrans, because we only use unicode for superscript.
            end if
         end if
         put htt after thehtmltext
      end if
   end repeat
   return thehtmltext
end formattedInput2

To test this you need the following controls on the card:

One visible input field called fldTest in Arial font size 18, containing the following text: x4+2x2y2+y4
One hidden field called fldHtmlEntity in Arial font
and one button with the following script:

Code: Select all

global superScriptArray, superScriptArrayTrans

on mouseUp
   set the htmlText of field fldTest to formattedInput2(the short id of field fldTest, "2,6,8,11")
end mouseUp

Save and close the stack, then open it again and press the button to do the htmlText formatting.

[Ledigimate]

Thanks you all for your suggestions.

I managed to rewrite the htmlText formatting function for LC8, and it no longer checks for native characters anywhere, so the function is significantly shorter:

Code: Select all

function formattedInput2 theFieldID, theExponents
   -- returns htmlText for the specified field, formatting the characters in the specified positions as exponents, i.e. superscript.
   -- theFieldID:  the short id of the field
   -- theExponents:  a list of the positions of the characters that should be converted to exponents, i.e. superscript.
   -- there MUST be an invisible field called fldHtmlEntity on every card that calls this function!
   local thehtmltext /*the function's return value*/, htt /*an htmlText chunk*/
   repeat with i = 1 to the length of field id theFieldID
      -- as of LC7, all text in fields are UNICODE.
      put codepointToNum(codepoint i of field id theFieldID) into tNum
      put (tNum is among the keys of superScriptArrayTrans) into tSuperScript
      put char i of field id theFieldID into tChar
      put tChar into field fldHtmlEntity
      if i is among the items of theExponents then -- character i should be an exponent, i.e. superscript
         if tSuperScript is true then -- leave it superscript
            put char 4 to -5 of the htmlText of field fldHtmlEntity into htt -- parse the html entity of the superscript character
         else -- make it superscript
            put numToCodepoint(superScriptArray[char i of field id theFieldID]) into field fldHtmlEntity
            put char 4 to -5 of the htmlText of field fldHtmlEntity into htt -- parse the html entity of the superscript character
         end if
      else -- character i should be normal (NOT an exponment), i.e. not superscipt
         if tSuperScript is true then -- make it normal
            put superScriptArrayTrans[tNum] into tChar
         end if
         if tChar is not among the chars of "abcdefghijklmnopqrstuvwxyz" then
            put htmlLiteral(tChar) into htt
         else
            put "<i>" & tChar & "</i>" into htt
         end if
      end if
      put htt after thehtmltext
   end repeat
   return thehtmltext
end formattedInput2

Kind regards,

Gerrie