Page 1 of 1
NULL character, cause of
Posted: Wed Dec 27, 2017 5:42 am
by montymay
I hope there is a simple explanation for some behavior that I do not understand. I am developing on the Windows platform. My script puts a text field with some interior text marked with braces into a variable and parses it with the following code:
Code: Select all
put the htmltext of fld "text" into tText
put offset("{", tText) into tLeftBracePos
put char 1 to tLeftBracePos-2 of tText into tText1
The script goes on to replace the text inside the braces with "FN" and finally gets the text from the right brace to the end of the text. Then the following statement put the text all back together:
Code: Select all
put tText1 & " FN " & tText2 into tNewNext
The script then writes tNewText to a text file with which I populate a text field, but there is an unwanted empty first line in the text field. When I open the text file with Notepad++, there is a null character in the first line. When I open it with Notepad, there is this: "<p>�</p>", but sometimes I see just Chinese characters! I don't want an empty first line. What is causing this result or how to I prevent it or how to I fix it? Thanks for any suggestions.
Monty May
Re: NULL character, cause of
Posted: Wed Dec 27, 2017 5:56 am
by dunbarx
Hi,
No idea what is in tText2, if that matters.
No empty line in the output though. On a Mac, though, and the output placed into an LC field. LC 8.1.8,
Craig Newman
Re: NULL character, cause of
Posted: Wed Dec 27, 2017 8:21 am
by montymay
Thanks for your reply. tText2 is the variable that contains all the text that follows the right brace character, which is used to mark the end of what is intended to be a footnote, from there to the last character in the text contained in the tText2 variable. As far as a I know, and I know very little about the details, the content of tText2 is not responsible for creating the empty first line. This behavior may be peculiar to Windows and doesn't happen on the Mac platform. I am thinking it has something to done with the code for line endings on Windows.
Re: NULL character, cause of
Posted: Wed Dec 27, 2017 11:16 am
by richmond62
This is probably a daft idea, but how about putting EMPTY
into your textfield before you populate it with your processed text?
Re: NULL character, cause of
Posted: Wed Dec 27, 2017 11:41 am
by [-hh]
You could try (if tLeftBracePos>0)
Code: Select all
put char 1 to tLeftBracePos-1 of tText into tText1
Also, for writing to an UTF-8 encoded file, you may need:
Code: Select all
put textEncode(fld "out","UTF-8") into url ("file:" & /path/to/fileout.txt)
Re: NULL character, cause of
Posted: Wed Dec 27, 2017 4:16 pm
by MaxV
You are working with htmltext, not with simple text, so please post an example to see what is inside it.
Re: NULL character, cause of
Posted: Thu Dec 28, 2017 10:45 am
by richmond62
I am interested as to why you headed this "NULL".
Re: NULL character, cause of
Posted: Fri Dec 29, 2017 11:53 pm
by montymay
Hello MaxV, Richmond62, -hh, and others
In response to Richmond62, I choose the word "NULL" as the description of the topic because when I populate a text field with the file that my script creates, it has an empty first line. When I open the file in Notepad++, it represents the empty line with a NULL symbol:

When I open a file created by my script in Notepad, I see this:
Regarding MaxV's reply, here is the entire script:
Code: Select all
## Assumption: the user has entered some text and marked footnote into field "text" by enclosing it with braces. This script will update the file so that the text appears with a footnote reference
## and the citation appears in a separate text file listing all the footnotes.
on mouseup
put empty into tNewText
local tText, tSkip = "0", tCount = "1", tFNCount = "1", tStart, tEnd
put the cTitle of me into tTitle
put "c:/Users/"&gComputer&"/Documents/DB/articles/" into tFolder
## (1) get the newly edited text that will be update the article
put the htmltext of fld "text" into tText
put tText into fld "text1"
## (2) get text after the new entry and its marked citation & put it into tText2
put offset("}", tText) into tRightBracePos
put char tRightBracePos+1 to -1 of tText into tText2
put tText2 into fld "text2"
## (3) get just the old text plus the user's entry in the body, but not the citation, and put it into tText1
put the htmltext of fld "text" into tText
put offset("{", tText) into tLeftBracePos
put char 1 to tLeftBracePos-2 of tText into tText1
put tText1 into fld "text1"
## (3) get the footnote and put it into tCitation and delete it the marked citation from tFile1
put char (tLeftBracePos+1) to (tRightBracePos-1) of tText into tCitation
put tCitation into fld "text3"
## (4) Put it all together
put tText1 & " FN " & tText2 into tNewText
put tNewText into fld "text1"
## (5) get the number of footnote references in the body
put FNnum("FN", tNewText) into tFNnum
## (6) Format the new unformatted footnote reference in the variable tNewText while updating the footnote
## reference numbers
set the caseSensitive to "true"
repeat tFNnum
get WordOffset("FN", tNewText, tSkip) --word
add it to tSkip -- put word number of FN
if "</sup>" is not in word tSkip of tNewText then --format & update one-word FN
put "<a><sup shift=""e&"-12""e&">FN"&tCount& "</sup></a>" into word tSkip of tNewText
put tCount & ") " & tCitation into tCitation
add 1 to tSkip --now three words
else put "<a><sup shift=" "e&"-12""e& ">FN"&tCount& "</sup></a>" into word (tSkip-1) to tSkip of tNewText --update three-word FN
add 1 to tCount
end repeat
## (7) Update the body of the text file and display it in the field "text"
open file tFolder&tTitle&".txt" for write
write tNewText to file tFolder&tTitle&".txt" at 0
close file tFolder&tTitle&".txt"
set the HTMLtext of fld "text" to it
##(8) Update the footnote file
[script omitted]
## (9) Diplay updated file in fld "text"
[script omiited]
end mouseup
function FNnum pSearch, pCont
put 0 into rFNnum
put 0 into tCharsToSkip
repeat
put offset(pSearch, pCont, tCharsToSkip) into tHit
if tHit = 0 then exit repeat
add tHit to tCharsToSkip
add 1 to rFNnum
end repeat
return rFNnum
end FNnum
The resulting field appears as follows:
I will continue studying the problem, following -hh's lead about encoding the text. My thanks to all.
Monty
Re: NULL character, cause of
Posted: Tue Jan 02, 2018 10:13 am
by montymay
Perhaps I have found the cause of the empty first line in the way I wrote the write to file command. My script read:
Code: Select all
open file tFolder&tTitle&".txt" for write
write tNewText to file tFolder&tTitle&".txt" at 0
close file tFolder&tTitle&".txt"
set the HTMLtext of fld "text" to it
When I deleted the "at 0" and simply stated "write tNewText to file tFolder&tTitle&".txt", the field populated as I wanted it! I have using 0" because "1" didn't work. The LC dictionary states: "The start specifies the character or byte position in the file where you want to begin writing. A positive number begins start characters
after the beginning of the file." The entry for "after" states: "[w[hen you use the after keyword, the current contents of the container . . . is not affected." So I am guessing that "0" is an illegal number and interpreted as a non-negative number. SInce the file was empty before it was written to, it wote to the second line. I'm just hobbyist programmer and so could be wrong, so please correct this entry so that I do not lead a reader astray.
Monty
Re: NULL character, cause of
Posted: Tue Jan 02, 2018 5:11 pm
by MaxV
You don't need anymore to open, just use
for reading txt files.
And
for writing.
Morevoer your code would be much simpler this way:
Code: Select all
replace "{" with ("<a><sup shift=""e&"-12""e&">FN§") in tText
replace "}" with "</sup></a>" in tText
put 0 into tCount
repeat for each char tChar in tText
if tChar = "§" then
add 1 to tCount
put tCount after tText2
else
put tChar after tText2
end if
end repeat
tText2 contains the new htmlText.
Re: NULL character, cause of
Posted: Wed Jan 03, 2018 8:47 pm
by montymay
Thanks, MaxV, for your post. Just two points. I was aware of the URL method but used the alternative because the URL method did not work for me when I used the following syntax:
Code: Select all
put URL "file:"&tFolder&tTitle&".txt" into fld "text"
After many tweaks the solution for me was putting the URL into parentheses! The Dictionary entry for "URL" does not say anything about the need for parentheses.
Code: Select all
put URL ("file:"&tFolderPath&tTitle&".txt") into fld "text"
Two, your script for creating a footnote reference is much simpler and shorter, and I want to use it now, but it does not increment the footnote references that follow the new statement and its footnote reference. I am working on a synthesis of my former method and yours, but if you see a solution, it would be much appreciated.
Monty
Re: NULL character, cause of
Posted: Thu Jan 04, 2018 2:42 pm
by MaxV
Probably you want this:
########CODE to copy and paste#######
on MouseUp
put the htmlText of field "text" into tText
#let's archive all footnotes
put tText into tText2
put 0 into tCount
repeat while matchtext(tText2,"(\{.*\})",tCite)
add 1 to tCount
put offset(tCite,tText2) into tskip
put char (tSkip + the number of chars of tCite + 1) to -1 of tText2 into tText2
put char 2 to -2 of tCite into tCite
put "<a name=FN" & tCount & " ><b>FN" & tCount & "</b>: " & tCite & "</a><br>" & return after tNotes
end repeat
#now we remove them
put replaceText(tText,"\{.*\}", "{}") into tText
#now we add the links
replace "{" with ("<sup><a href=#FN§>FN§") in tText
replace "}" with "</a></sup>" in tText
put 0 into tCount
put empty into tText2
repeat for each char tChar in tText
if tChar = "§" then
add 1 to tCount
put tCount after tText2
else
put tChar after tText2
end if
end repeat
put tText2 & tNotes into URL "file:myTest.html"
launch URL "myTest.html"
end MouseUp
#####END OF CODE generated by this livecode app: http://tinyurl.com/j8xf3xq with livecode 9.0.0-dp-11#####
Re: NULL character, cause of
Posted: Fri Jan 05, 2018 8:31 am
by Thierry
montymay wrote: ↑Wed Jan 03, 2018 8:47 pm
...., but if you see a solution, it would be much appreciated.
Hi Monty,
my take on this:
On this screenshot; top-left is the input text,
bottom-left is the generated output and right the output in my browser.
and here is the code:
Code: Select all
function buildFootNotes Txt
local Notes, fnCounter, fnText
local fnLinkTemplate = "<sup><a href=#FN[[fnCounter]]>FN[[fnCounter]]</a></sup> "
local fnTextTemplate = "<a name=FN[[fnCounter]]><b>FN[[fnCounter]]</b>: [[fnText]]</a><br>"
put "<br /><br />_____ notes: _____<br />" into Notes
repeat while matchChunk( Txt, "(?x) ( { \s* (.*?) \s* } )", p1, p2, p3, p4)
add 1 to fnCounter
put char p3 to p4 of Txt into fnText
put merge( fnLinkTemplate) into char p1 to p2 of Txt
put merge( fnTextTemplate) after Notes
end repeat
return Txt & Notes
end buildFootNotes
and the script button:
Code: Select all
on mouseUp
local myHtmlFile = "file:/Users/xxxx/testBuildFootnotes.html"
put the htmlText of field "fInput" into tText
set the text of field "fResult" to buildFootNotes( tText)
put the text of field "fResult" into URL myHtmlFile
launch URL myHtmlFile
end mouseUp
HTH,
Thierry