reading data from an active log file

[magice]

One of the apps I am currently working on will have to read a log file that is being generated by another program. It will have to find log entries that contain specific phrases and then pick out words from those lines. It will have to read till the end of the file, then remember where it left off so that in a few seconds when it reopens the file it only reads what has been added since the last read. Here is a sample of how the log file is written:


(1296699867)[Wed Feb 02 21:24:27 2011] Line I want to ignore
(1296699869)[Wed Feb 02 21:24:29 2011] Line I want to ignore
(1296699875)[Wed Feb 02 21:24:35 2011] data I need to retrieve Specific phrase I need to search for
(1296699889)[Wed Feb 02 21:24:49 2011] Line I want to ignore

I believe I can use the log entry numbers to the left to remember my place, and I remember reading other posts that explained how to pick out a specific line from a text file. At this point my only experience with making an app that reads from a text file has been reading till the eof, then dumping the whole file into a variable. Can anyone point me to a tutorial that deals with more advanced parsing techniques?

[magice]

I have been experimenting all day with everything I have found in the documentation to achieve the goal of my above described post. I have had no luck at all. It seems that every command to start reading in a text file at a specific location requires the line number in the text file. What I seem to need and can't find is a command that will search the entire document for a word and return back the line number of the line in which the word was found. I suppose i could also make it work if I had a way to get the line number of the last line in the file and with each file read start at the previous read's last line number. I must also be able to specify my search criteria and file location in variables. any suggestions would be greatly appreciated I'm starting to think in circles and it's making my brain hurt.

[bangkok]

I suppose i could also make it work if I had a way to get the line number of the last line in the file and with each file read start at the previous read's last line number. I must also be able to specify my search criteria and file location in variables. any suggestions would be greatly appreciated I'm starting to think in circles and it's making my brain hurt.

I think you found the solution yourself : keep a track of the number of the last line that has been processed, right ?

Let say :
-first read
put the number of lines of myLogs into theLine

store this value in a simple text file.

-and after for further reading, read the value, read the file, and substract the value from the new total number of lines, that will give you the line number from which you have to process the data.

etc.

As to parse the content of the file, you should have a look at the lineoffset function, or even better itemoffset.

[magice]

-and after for further reading, read the value, read the file, and substract the value from the new total number of lines, that will give you the line number from which you have to process the data.

etc.

As to parse the content of the file, you should have a look at the lineoffset function, or even better itemoffset.

The lineoffset function is what I spent most of my day yesterday fighting. For some reason I keep getting "Handler not found" errors. It is the same problem I am having with another app where I need to use the revZip functions. It could be that I am using an older version of rev (3.0) on a library computer, but unfortunately being a poor student working my way through school makes buying a new copy for myself difficult, and I don't have admin rights on this computer to do an update. in any case, after hours of trial and error, the following code works to remember my position:

Code: Select all

on mouseUp
   global tLogFileLoc
   global tLogFileMarker
   set the disabled of me to true
   open file tLogFileLoc for read
   answer tLogFileMarker
   read from file tLogFileLoc at tLogFileMarker until eof
   add the number of characters in it to tLogFileMarker
   answer it
   close file tLogFileLoc
   set the disabled of me to false
end mouseUp

Now my problem is searching the lines in "it" for specific strings and extracting the data I need. for example, if one of the lines of the text file says:
(1296699772)[Wed Feb 02 21:22:52 2011] Treva has joined the group.
I need to be able to search the entire body of text in the above "it" variable (which could be several hundred lines)for "has joined the group" and have it return either "Treva" or the entire line so I can dissect it and get "Treva"

[magice]

Ok, I found a workaround If i put my chunk of the log into an invisible text area field, then I can use the find and foundline commands to do what I want. It might not be the most elegant solution but it works.

[BvG]

try lineoffset

[bangkok]

Hum.. I think you are mixing several issues and questions there.

First : the "Handler not found" and RevZip issue... I don't really understand.
You should make your tests in a plain new stack, without using special functions/libraries.

Second : the lineoffset will lead nowhere for a very simple reason : on each line, you have fixed texts (for instance : "has joined the group") and variable texts ("TREVA").

So this is why, I think you should keep it simple :

-read the file
-put it into a variable
-and then make a repeat structure, in which you are going to look for the fixed texts. even with thousands lines, it should be okay.
-if true, put this line in another variable, on which you will do further work

So something like this :

Code: Select all

on mouseUp
   global tLogFileLoc
   global tLogFileMarker
   set the disabled of me to true
   open file tLogFileLoc for read
   read from file tLogFileLoc at tLogFileMarker until eof

put it into toBeAnalysed

   add the number of characters in toBeAnalysed to tLogFileMarker
   answer toBeAnalysed
   close file tLogFileLoc
   set the disabled of me to false

put empty into secondList

repeat with i = 1 to the number of lines of toBeAnalysed
if line i of toBeAnalysed contains "has joined the group" then put line i of toBeAnalysed & CR after secondList
end repeat

-- now secondList contains all the lines with "has joined the group".... so now you can perform further analysis/actions on this variable
end mouseUp