Reading and writing to a text file

[anmldr]

Should I open a text file, read it, close it, open it again, write to it and close it again? The wording is "open file...for read" and "open file...for write".

Can you open a file for reading AND writing or should I open (read)/close/open (write)/close every time that I want to alter the file?

TIA,
Linda

[SparkOut]

If you use "update" rather than either "read" or "write" then that covers both
You may (or may not) want to preserve the format by specifying "for text update" or "for binary update" too. Or such combination of choices as is appropriate.

[anmldr]

Thank you. I will be able to find those in the dictionary now.

Linda

[bogs]

Well, you can certainly do that, open / update / read / etc a file, but I think the simplest way would be to just put your data into the file itself, unless you are adding dribs and drabs of text to the file.

For example, if your writing a field to a text file, it would look like

Code: Select all

put field "myField" into url("File:" & "the path to your file")

[Klaus]

Hi Linda,

unless you are dealing with BIG text files > 10 MB, like Server log files, you could use the simple URL systax as Craig already showed you.
You can even treat this like text in a LC field and do something like:

Code: Select all

...
put "path/to/your/file/here.txt" into tFile
put CR & "New last line" AFTER url("file:" & tFile)
...
put "New first line" & CR BEFORE url("file:" & tFile)
...
put "some text in the middle" after line 10 of url("file:" & tFile)
...

etc.
You get the picture.


Best

Klaus

[bogs]

I can't help but wonder why the 'open for read/write/update' even exists at this point for text files in Lc. Is there a legitimate case for it, other than that other languages have it?

By 'legitimate case', I mean is there something you can do with it that you can not accomplish just putting text into a directly.

[SparkOut]

If I understand correctly, the url ("file:...") method reads the entire file into memory, does the operation, and saves the entire file out every time it is updated. So one word, efficiency. For smaller files, this is not really humanly noticeable.
Additionally, in the case of very large files (2GB upwards, so not even that astronomic) it can also fail to load at all, or even crash the app/LiveCode completely.
With the "open file" method you can read in chunks and process in a less memory intensive manner. It was something that I was forced to do when opening and converting a file of telemetry data for one application.

[bogs]

That is interesting. Trying to remember the last time I saw a 2 GB text file hahaha, aside from (as Klaus mentioned) server files, I'm drawing a blank. However, now I'm trying to figure out how loading a file with 'url' is going to load an entire file into memory, but opening it for 'read' or 'update' is not.

BTW, you forgot to mention this little gem -
Important! : Before opening any file for writing, remember to back up the file contents as LiveCode will erase them even if you do not write to the file after opening it.

Also, in the Dictionary, such an important distinction isn't made at all as far as I could find. In the Dictionary, it says that url is an alternative to open.

[SparkOut]

alternative != synonym

Or more LiveCodeish, "alternative" <> "synonym"

My use case for a file that large was telemetry data for a racing car. There could be plenty of industrial examples.

It's a very valid point though, that for writing to (or updating) a file with lots of new little pieces of data repeatedly over time, the efficiency overhead of the url ("file:...") method could be a problem. With "open file for..." I believe it creates a buffer with a pointer to the current data point in the file. Probably also hands off some of the work to the OS APIs.

And open for write will in essence, start a new file buffer with the pointer set at the beginning of the file. In other cases, open file for update would be what is needed.

[bogs]

LOL!

"Synonym" never came out of what I said haha. Just to be clear, This is what I was referring too:

Tip: As an alternative to the open file, read from file, and write to file commands, you can also use the URL keyword with get, put, and other commands to access the contents of a file.

I still don't see how one would not open the whole file (as in for append, update, etc), the dictionary states it does as much, in some cases in a fairly dangerous way (such as open for write, as you say).

Ah well.

[SparkOut]

I just edited above to answer that point. I think the buffer pretty much does get assistance from the OS.
I think of it in terms more like a socket.

And I will also say, if I can take the easy option and put something into url("file:...") then I will do that before I
would open a file for whatever. Convenience usually trumps efficiency, unless there's a particular reason.

[bogs]

Playing around with creating large text files heh. You can tell i really have nothing else to do for the next 30 minutes haha.

[jacque]

Trivia: if you open a file without specifying the method the default is to open it for update. This allows both reading and writing:

Code: Select all

open file "my file.txt"

There are lots of times where it's useful to use "open file" rather than the URL container. I've used "open file for append" to write to a log file for example. The file is opened once and left open while the app runs, then closed later. Every log entry can be written to the end of the file without the overhead of loading the entire content of the file into RAM. It's also faster, since log files can become large and there's no need for the engine to search for the end since the pointer is already there.

Another way I use open file is when I only want to examine a small part of the file. For example, if I want to determine the file format of an image I only need to read the first few bytes. It's more efficient to open the file for binary read, get the first bytes, and close the file without loading megabytes of data into memory. I use this method to identify gz files too.

Sometimes finer control over over the incoming data is required, that's where it's helpful to be able to read for only a certain amount at a time or until you get the piece your looking for. Reading a file for one line and then processing the line is efficient and fast, sort of similar to "repeat for each". On mobile where memory is limited it may also avoid a crash or freeze.

[AxWald]

Bogs,

another example for "open file":

You're processing phat data, line- wise, using an output variable:

Code: Select all

   repeat for each line L in myData
      put "'" & item 5 of L & "'" & tab & round(item 11 of L,2) & tab & ...  & CR after myVar
   end repeat

It's easy, fast & cozy - until myVar reaches a certain size. Then the speed will degrade heavily. So you'll use:

Code: Select all

   open file "tempfile.txt" for append
   repeat for each line L in myData
      write "'" & item 5 of L & "'" & tab & round(item 11 of L,2) & tab & ...  & CR to file "tempfile.txt"
   end repeat
   close file "tempfile.txt"
   put URL "file:tempfile.txt" into myVar

Now LC doesn't have to find the end of myVar after each iteration anymore, it just throws the new line to the OS. There another process takes care of it. This results in a dramatic speed gain.

Btw., even another scenario:
You want to read from a file that another process has locked & writes to - open file xy for read happily does this.

Have fun!

[bogs]

I didn't really get back to this like I wanted to (the day had started so promising ), but I did take the time to test out 6 or so different VERY large file methods. I can see why you'd need open in certain circumstances now.

Interesting side learning to all this, I found quite a few text editors can't open a file that large either