Page 1 of 1
Reading and writing to a text file
Posted: Fri Jun 05, 2020 9:37 pm
by anmldr
Should I open a text file, read it, close it, open it again, write to it and close it again? The wording is "open file...for read" and "open file...for write".
Can you open a file for reading AND writing or should I open (read)/close/open (write)/close every time that I want to alter the file?
TIA,
Linda
Re: Reading and writing to a text file
Posted: Fri Jun 05, 2020 9:52 pm
by SparkOut
If you use "update" rather than either "read" or "write" then that covers both
You may (or may not) want to preserve the format by specifying "for text update" or "for binary update" too. Or such combination of choices as is appropriate.
Re: Reading and writing to a text file
Posted: Fri Jun 05, 2020 10:15 pm
by anmldr
Thank you. I will be able to find those in the dictionary now.
Linda
Re: Reading and writing to a text file
Posted: Fri Jun 05, 2020 11:00 pm
by bogs
Well, you can certainly do that, open / update / read / etc a file, but I think the simplest way would be to just put your data into the file itself, unless you are adding dribs and drabs of text to the file.
For example, if your writing a field to a text file, it would look like
Code: Select all
put field "myField" into url("File:" & "the path to your file")
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 8:25 am
by Klaus
Hi Linda,
unless you are dealing with BIG text files > 10 MB, like Server log files, you could use the simple URL systax as Craig already showed you.
You can even treat this like text in a LC field and do something like:
Code: Select all
...
put "path/to/your/file/here.txt" into tFile
put CR & "New last line" AFTER url("file:" & tFile)
...
put "New first line" & CR BEFORE url("file:" & tFile)
...
put "some text in the middle" after line 10 of url("file:" & tFile)
...
etc.
You get the picture.
Best
Klaus
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 9:40 am
by bogs
I can't help but wonder why the 'open for read/write/update' even exists at this point for text files in Lc. Is there a legitimate case for it, other than that other languages have it?
By 'legitimate case', I mean is there something you can do with it that you can not accomplish just putting text into a directly.
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 9:58 am
by SparkOut
If I understand correctly, the url ("file:...") method reads the entire file into memory, does the operation, and saves the entire file out every time it is updated. So one word, efficiency. For smaller files, this is not really humanly noticeable.
Additionally, in the case of very large files (2GB upwards, so not even that astronomic) it can also fail to load at all, or even crash the app/LiveCode completely.
With the "open file" method you can read in chunks and process in a less memory intensive manner. It was something that I was forced to do when opening and converting a file of telemetry data for one application.
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 10:19 am
by bogs
That is interesting. Trying to remember the last time I saw a 2 GB text file hahaha, aside from (as Klaus mentioned) server files, I'm drawing a blank. However, now I'm trying to figure out how loading a file with 'url' is going to load an entire file into memory, but opening it for 'read' or 'update' is not.
BTW, you forgot to mention this little gem -
Important! : Before opening any file for writing, remember to back up the file contents as LiveCode will erase them even if you do not write to the file after opening it.
Also, in the Dictionary, such an important distinction isn't made at all as far as I could find. In the Dictionary, it says that url is an alternative to open.
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 10:22 am
by SparkOut
alternative != synonym
Or more LiveCodeish, "alternative" <> "synonym"
My use case for a file that large was telemetry data for a racing car. There could be plenty of industrial examples.
It's a very valid point though, that for writing to (or updating) a file with lots of new little pieces of data repeatedly over time, the efficiency overhead of the url ("file:...") method could be a problem. With "open file for..." I believe it creates a buffer with a pointer to the current data point in the file. Probably also hands off some of the work to the OS APIs.
And open for write will in essence, start a new file buffer with the pointer set at the beginning of the file. In other cases, open file for update would be what is needed.
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 10:33 am
by bogs
LOL!
"Synonym" never came out of what I said haha. Just to be clear, This is what I was referring too:
Tip: As an alternative to the open file, read from file, and write to file commands, you can also use the URL keyword with get, put, and other commands to access the contents of a file.
I still don't see how one would not
open the whole file (as in for append, update, etc), the dictionary states it does as much, in some cases in a fairly dangerous way (such as open for write, as you say).
Ah well.
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 10:38 am
by SparkOut
I just edited above to answer that point. I think the buffer pretty much does get assistance from the OS.
I think of it in terms more like a socket.
And I will also say, if I can take the easy option and put something into url("file:...") then I will do that before I
would open a file for whatever. Convenience usually trumps efficiency, unless there's a particular reason.
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 11:06 am
by bogs
Playing around with creating large text files heh. You can tell i really have nothing else to do for the next 30 minutes haha.
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 4:56 pm
by jacque
Trivia: if you open a file without specifying the method the default is to open it for update. This allows both reading and writing:
There are lots of times where it's useful to use "open file" rather than the URL container. I've used "open file for append" to write to a log file for example. The file is opened once and left open while the app runs, then closed later. Every log entry can be written to the end of the file without the overhead of loading the entire content of the file into RAM. It's also faster, since log files can become large and there's no need for the engine to search for the end since the pointer is already there.
Another way I use open file is when I only want to examine a small part of the file. For example, if I want to determine the file format of an image I only need to read the first few bytes. It's more efficient to open the file for binary read, get the first bytes, and close the file without loading megabytes of data into memory. I use this method to identify gz files too.
Sometimes finer control over over the incoming data is required, that's where it's helpful to be able to read for only a certain amount at a time or until you get the piece your looking for. Reading a file for one line and then processing the line is efficient and fast, sort of similar to "repeat for each". On mobile where memory is limited it may also avoid a crash or freeze.
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 7:07 pm
by AxWald
Bogs,
another example for "open file":
You're processing phat data, line- wise, using an output variable:
Code: Select all
repeat for each line L in myData
put "'" & item 5 of L & "'" & tab & round(item 11 of L,2) & tab & ... & CR after myVar
end repeat
It's easy, fast & cozy - until myVar reaches a certain size. Then the speed will degrade heavily. So you'll use:
Code: Select all
open file "tempfile.txt" for append
repeat for each line L in myData
write "'" & item 5 of L & "'" & tab & round(item 11 of L,2) & tab & ... & CR to file "tempfile.txt"
end repeat
close file "tempfile.txt"
put URL "file:tempfile.txt" into myVar
Now LC doesn't have to find the end of myVar after each iteration anymore, it just throws the new line to the OS. There another process takes care of it. This results in a dramatic speed gain.
Btw., even another scenario:
You want to read from a file that another process has locked & writes to - open file xy for read happily does this.
Have fun!
Re: Reading and writing to a text file
Posted: Sat Jun 06, 2020 11:17 pm
by bogs
I didn't really get back to this like I wanted to (the day had started so promising
), but I did take the time to test out 6 or so different VERY large file methods. I can see why you'd need open in certain circumstances now.
Interesting side learning to all this, I found quite a few text editors can't open a file that large either
