Page 1 of 1

NULL character, cause of

Posted: Wed Dec 27, 2017 5:42 am
by montymay
I hope there is a simple explanation for some behavior that I do not understand. I am developing on the Windows platform. My script puts a text field with some interior text marked with braces into a variable and parses it with the following code:

Code: Select all

   
     put the htmltext of fld "text" into tText
     put offset("{", tText) into tLeftBracePos
     put char 1 to tLeftBracePos-2 of tText into tText1
     
The script goes on to replace the text inside the braces with "FN" and finally gets the text from the right brace to the end of the text. Then the following statement put the text all back together:

Code: Select all

put tText1 & " FN " & tText2 into tNewNext
The script then writes tNewText to a text file with which I populate a text field, but there is an unwanted empty first line in the text field. When I open the text file with Notepad++, there is a null character in the first line. When I open it with Notepad, there is this: "<p>&#0;</p>", but sometimes I see just Chinese characters! I don't want an empty first line. What is causing this result or how to I prevent it or how to I fix it? Thanks for any suggestions.

Monty May

Re: NULL character, cause of

Posted: Wed Dec 27, 2017 5:56 am
by dunbarx
Hi,

No idea what is in tText2, if that matters.

No empty line in the output though. On a Mac, though, and the output placed into an LC field. LC 8.1.8,

Craig Newman

Re: NULL character, cause of

Posted: Wed Dec 27, 2017 8:21 am
by montymay
Thanks for your reply. tText2 is the variable that contains all the text that follows the right brace character, which is used to mark the end of what is intended to be a footnote, from there to the last character in the text contained in the tText2 variable. As far as a I know, and I know very little about the details, the content of tText2 is not responsible for creating the empty first line. This behavior may be peculiar to Windows and doesn't happen on the Mac platform. I am thinking it has something to done with the code for line endings on Windows.

Re: NULL character, cause of

Posted: Wed Dec 27, 2017 11:16 am
by richmond62
This is probably a daft idea, but how about putting EMPTY
into your textfield before you populate it with your processed text?

Re: NULL character, cause of

Posted: Wed Dec 27, 2017 11:41 am
by [-hh]
You could try (if tLeftBracePos>0)

Code: Select all

put char 1 to tLeftBracePos-1 of tText into tText1
Also, for writing to an UTF-8 encoded file, you may need:

Code: Select all

put textEncode(fld "out","UTF-8") into url ("file:" & /path/to/fileout.txt)

Re: NULL character, cause of

Posted: Wed Dec 27, 2017 4:16 pm
by MaxV
You are working with htmltext, not with simple text, so please post an example to see what is inside it.

Re: NULL character, cause of

Posted: Thu Dec 28, 2017 10:45 am
by richmond62
I am interested as to why you headed this "NULL".

Re: NULL character, cause of

Posted: Fri Dec 29, 2017 11:53 pm
by montymay
Hello MaxV, Richmond62, -hh, and others

In response to Richmond62, I choose the word "NULL" as the description of the topic because when I populate a text field with the file that my script creates, it has an empty first line. When I open the file in Notepad++, it represents the empty line with a NULL symbol:
Image
When I open a file created by my script in Notepad, I see this:
Image

Regarding MaxV's reply, here is the entire script:

Code: Select all

## Assumption: the user has entered some text and marked footnote into field "text" by enclosing it with braces. This script will update the file so that the text appears with a footnote reference 
## and the citation appears in a separate text file listing all the footnotes.

on mouseup
     put empty into tNewText
     local tText, tSkip = "0", tCount = "1", tFNCount = "1", tStart, tEnd
     put the cTitle of me into tTitle
     put "c:/Users/"&gComputer&"/Documents/DB/articles/" into tFolder  
     
     ## (1) get the newly edited text that will be update the article
     put the htmltext of fld "text" into tText
     put tText into fld "text1"
     
     
     ## (2) get text after the new entry and its marked citation & put it into tText2
     put offset("}", tText) into tRightBracePos
     put char tRightBracePos+1 to -1 of tText into tText2
     put tText2 into fld "text2"
     
     ## (3) get just the old text plus the user's entry in the body, but not the citation, and put it into tText1
     put the htmltext of fld "text" into tText
     put offset("{", tText) into tLeftBracePos
     put char 1 to tLeftBracePos-2 of tText into tText1
     put tText1 into fld "text1"
     
     ## (3) get the footnote and put it into tCitation and delete it the marked citation from tFile1
     put char (tLeftBracePos+1) to (tRightBracePos-1) of tText into tCitation
     put tCitation into fld "text3"
     
     ## (4) Put it all together
     put tText1 & " FN " & tText2 into tNewText
     put tNewText into fld "text1"
     
     
     ## (5) get the number of footnote references in the body
     put FNnum("FN", tNewText) into tFNnum
     
     ## (6) Format the new unformatted footnote reference in the variable tNewText while updating the footnote
     ## reference numbers
     set the caseSensitive to "true" 
     repeat tFNnum
          get WordOffset("FN", tNewText, tSkip)  --word
          add it to tSkip  -- put word number of FN 
          if "</sup>" is not in word tSkip of tNewText then  --format & update one-word FN
               put "<a><sup shift="&quote&"-12"&quote&">FN"&tCount& "</sup></a>" into word tSkip of tNewText
               put tCount & ") " & tCitation into tCitation
               add 1 to tSkip  --now three words
          else put "<a><sup shift=" &quote&"-12"&quote& ">FN"&tCount& "</sup></a>" into word (tSkip-1) to tSkip of tNewText  --update three-word FN
          add 1 to tCount
     end repeat
     
     ## (7) Update the body of the text file and display it in the field "text"
     
     open file tFolder&tTitle&".txt" for write
     write tNewText to file tFolder&tTitle&".txt" at 0
     close file tFolder&tTitle&".txt"
     set the HTMLtext of fld "text" to it
     
     ##(8) Update the footnote file
    [script omitted]     
     
     ## (9) Diplay updated file in fld "text"
     [script omiited]     
end mouseup

   function FNnum pSearch, pCont
     put 0 into rFNnum
     put 0 into tCharsToSkip
     repeat
          put offset(pSearch, pCont, tCharsToSkip) into tHit
          if tHit = 0 then exit repeat
          add tHit to tCharsToSkip
          add 1 to rFNnum
     end repeat
     return rFNnum
end FNnum
The resulting field appears as follows:
Image

I will continue studying the problem, following -hh's lead about encoding the text. My thanks to all.

Monty

Re: NULL character, cause of

Posted: Tue Jan 02, 2018 10:13 am
by montymay
Perhaps I have found the cause of the empty first line in the way I wrote the write to file command. My script read:

Code: Select all

     open file tFolder&tTitle&".txt" for write
     write tNewText to file tFolder&tTitle&".txt" at 0
     close file tFolder&tTitle&".txt"
     set the HTMLtext of fld "text" to it
When I deleted the "at 0" and simply stated "write tNewText to file tFolder&tTitle&".txt", the field populated as I wanted it! I have using 0" because "1" didn't work. The LC dictionary states: "The start specifies the character or byte position in the file where you want to begin writing. A positive number begins start characters after the beginning of the file." The entry for "after" states: "[w[hen you use the after keyword, the current contents of the container . . . is not affected." So I am guessing that "0" is an illegal number and interpreted as a non-negative number. SInce the file was empty before it was written to, it wote to the second line. I'm just hobbyist programmer and so could be wrong, so please correct this entry so that I do not lead a reader astray.

Monty

Re: NULL character, cause of

Posted: Tue Jan 02, 2018 5:11 pm
by MaxV
You don't need anymore to open, just use

Code: Select all

put URL "file:..." into myVar
for reading txt files.

And

Code: Select all

put myVar into  URL "file:..." 
for writing.

Morevoer your code would be much simpler this way:

Code: Select all

replace "{" with ("<a><sup shift="&quote&"-12"&quote&">FN§") in tText
replace "}" with "</sup></a>" in tText
put 0 into tCount
repeat for each char tChar in  tText
if tChar = "§" then 
    add 1 to tCount
    put tCount after tText2
  else
    put tChar after tText2
  end if
end repeat
tText2 contains the new htmlText.

Re: NULL character, cause of

Posted: Wed Jan 03, 2018 8:47 pm
by montymay
Thanks, MaxV, for your post. Just two points. I was aware of the URL method but used the alternative because the URL method did not work for me when I used the following syntax:

Code: Select all

put URL "file:"&tFolder&tTitle&".txt" into fld "text"
After many tweaks the solution for me was putting the URL into parentheses! The Dictionary entry for "URL" does not say anything about the need for parentheses.

Code: Select all

put URL ("file:"&tFolderPath&tTitle&".txt") into fld "text"
Two, your script for creating a footnote reference is much simpler and shorter, and I want to use it now, but it does not increment the footnote references that follow the new statement and its footnote reference. I am working on a synthesis of my former method and yours, but if you see a solution, it would be much appreciated.

Monty

Re: NULL character, cause of

Posted: Thu Jan 04, 2018 2:42 pm
by MaxV
Probably you want this:

########CODE to copy and paste#######
on MouseUp
put the htmlText of field "text" into tText
#let's archive all footnotes
put tText into tText2
put 0 into tCount
repeat while matchtext(tText2,"(\{.*\})",tCite)
add 1 to tCount
put offset(tCite,tText2) into tskip
put char (tSkip + the number of chars of tCite + 1) to -1 of tText2 into tText2
put char 2 to -2 of tCite into tCite
put "<a name=FN" & tCount & " ><b>FN" & tCount & "</b>: " & tCite & "</a><br>" & return after tNotes
end repeat
#now we remove them
put replaceText(tText,"\{.*\}", "{}") into tText
#now we add the links
replace "{" with ("<sup><a href=#FN§>FN§") in tText
replace "}" with "</a></sup>" in tText
put 0 into tCount
put empty into tText2
repeat for each char tChar in tText
if tChar = "§" then
add 1 to tCount
put tCount after tText2
else
put tChar after tText2
end if
end repeat
put tText2 & tNotes into URL "file:myTest.html"
launch URL "myTest.html"
end MouseUp
#####END OF CODE generated by this livecode app: http://tinyurl.com/j8xf3xq with livecode 9.0.0-dp-11#####

Re: NULL character, cause of

Posted: Fri Jan 05, 2018 8:31 am
by Thierry
montymay wrote:
Wed Jan 03, 2018 8:47 pm
...., but if you see a solution, it would be much appreciated.
Hi Monty,

my take on this:

On this screenshot; top-left is the input text,
bottom-left is the generated output and right the output in my browser.

sunnY- 2018-01-05 à 08.15.52.png
and here is the code:

Code: Select all

function buildFootNotes Txt
   local  Notes, fnCounter, fnText
   local fnLinkTemplate =  "<sup><a href=#FN[[fnCounter]]>FN[[fnCounter]]</a></sup> "
   local fnTextTemplate = "<a name=FN[[fnCounter]]><b>FN[[fnCounter]]</b>: [[fnText]]</a><br>"
   put "<br /><br />_____  notes:  _____<br />" into Notes
   repeat while matchChunk( Txt, "(?x) ( { \s* (.*?) \s* }  )", p1, p2, p3, p4) 
      add 1 to fnCounter
      put char p3 to p4 of Txt into fnText
      put merge( fnLinkTemplate) into char p1 to p2 of Txt
      put  merge( fnTextTemplate)  after Notes
   end repeat
   return Txt  & Notes
end buildFootNotes
and the script button:

Code: Select all

on mouseUp
   local myHtmlFile =  "file:/Users/xxxx/testBuildFootnotes.html"
   put the htmlText of field "fInput" into tText
   set the text of  field "fResult" to  buildFootNotes( tText)
   put  the text of  field "fResult" into URL myHtmlFile
   launch URL myHtmlFile
end mouseUp
HTH,

Thierry