How can I speed up this code?

[FourthWorld]

Thank you for posting the revised handler, but please understand that any rewrite cannot be checked for either accuracy or performance enhancement unless I can actually run it, and to do that I would need some data.

As I noted, It need not be the actual data your project uses; anything structured similarly and of roughly equivalent size will do.

[dunbarx]

It matters not that another field is used as some sort of intermediary. Load fields just a few times as needed, do all processing in variables, whether ordinary ones as I suggested, or arrays as Sturgis did.

I would think that either option would run through 150 lines in a millisecond.

Craig Newman

[sturgis]

In order for me to get a quick handle on using styledtext to do things, I've simplified your script some, but the basics will be the same.

Here is a script that should be close to what you want and its very very fast.

Code: Select all

on mouseUp
   put 1 into tLineCount -- line counter
   set the itemdelimiter to tab 
   repeat for each line tLine in field 1 -- grab each line in turn from the field

-- shove the line into a styledtext array variable
-- this means that we're putting the contents of tLine into an array with keys pointing to the line number, 
--designating it as a character run keyed to 1 (could have multiple runs)
--keyed indicating it is the text of the line
      put tLine into tStyleA[tlineCount]["runs"][1]["text"] 

-- do whatever checks you need to determine color etc..
      if item 4 of tLine > 0 then
         if item 4 of tLine > 3 then
-- To set the text style for the matching criteria set up another array branch for the same line
-- this puts the color designation into the array keyed again by linecount
-- but then by style, then by the type of styling to be done. I suspect "red" and "yellow" could have been used here too but haven't tried
            put "0,255,0" into tStyleA[tLineCount]["style"]["backgroundColor"]
         else
            put "255,0,0" into tStyleA[tLineCount]["style"]["backgroundColor"]
         end if
      else
-- if there was no match, since you want to empty the lines that don't match, shove empty into the text run
         put empty into tStyleA[tlineCount]["runs"][1]["text"]
      end if
      add 1 to tLineCount -- increment the counter
   end repeat
-- the array now contains both the text, and the styling for the field
-- so just set the styledtext of the field to the built up array. 
   set the styledtext of field 1 to tStyleA   
end mouseUp

10k lines takes about 100ms to process.

[dunbarx]

I just tried your original code. It worked in a flash. I upped the line count to 1500, and it still took under a second.

All that field reading and writing stuff is still valid, but the instance at hand should not take so long.

Something else going on here?

Craig Newman

[user#606]

Hi Richard,
I have ziped a stack consisting of Table 1 with data and the button that holds the code.
The button is just for testing and with the final code, it will be put into a background process to refresh once a minute.

Hi Sturgis, HTML is not an option in this case. The fact that there may only ever be a few lines, with coloured backgrounds, means the line background method is not the problem.

I had explored the posibility of having a separate table with just the few lines that will be affected, indexed to the main table field. That would be impractical in this case, because the line positions in the main table frequently change, due to user sorting. One table would no longer have the address in the other.

I have already moved some processing to other user related actions, reducing the time in the critical code.

[FourthWorld]

Hi Richard,
I have ziped a stack consisting of Table 1 with data and the button that holds the code.

Thanks. Did you mean to post it here? It wasn't attached.

[sturgis]

Not sure what html has to do with my example. But if it won't work for you no biggy. Still might pay to look at how the styledtext array works. Its useful!

The fact that there may only ever be a few lines, with coloured backgrounds, means the line background method is not the problem.

No, the problem is finding a more efficient way to cycle through the lines of the field to locate which ones should have a different background color.

looping through a field is slow when using line number references because each loop it has to cycle from line 1 to whatever line is current. a for each loop instead is like using a pointer. Each loop it indexes forward 1 more line and shoves the contents into the variable you designate and doesn't restart at 1 each loop so is MUCH faster. Cycle through with for each, figure out what you need to do with each line and do it, while building up the new changed data, then put it back into the field. (or in the case of the styledtext array, set styledtext of the field to the array that was built up) Which ends up being the same as using htmltext in the end. (if you: set the backgroundcolor of line 1 of field "myField" to "red" , and then get the htmltext of the field there will be an html tag setting the background color for that text.) Its all a matter of how you get there, the destination is the same.

[user#606]

I emailed direct

[FourthWorld]

I emailed direct

Thanks, but although I did get your first email I didn't get the one with the attachment. I've since whitelisted your address if you want to resend, but perhaps better still you could post it here as an attachment so everyone can test with the same sample data.

[user#606]

I have removed sensitive data
the central button is wher the problem code is.
The side button is to put rubbish into private fields.
Item 11 holds hours, item 12 holds minutes, item 13 holds the trigger to colour line or not.
There are always just a few triggers at any one time, never more than a dozen I would guess.

[FourthWorld]

Here's the revised code:

Code: Select all

on mouseUp
   put the millisecs into t
   --
   local NumLines, LineCount,Temp,Temp1
   set itemdelimiter to tab
   set the twelveHourTime to false
   set the useSystemDate to true
   put the time into Temp1
   convert Temp1 to seconds
   put fld "table 1" into tData
   put empty into tNuData
   put 0 into LineCount
   repeat for each line tLine in tData
      get item 13 of tLine
      if  it > 0 then
         put "<font color=red>" & item 6 of tLine &"</font>" into item 6 of tLine
         if  it  > Temp1 then
            put "<p bgColor=yellow>" & tLine & "</p>"&cr after tNuData
         else
            put "<p bgColor=red>" & tLine & "</p>"&cr after tNuData
         end if
      else 
         put empty into item 13 of tLine
         put "<p>" & tLine &"</p>" &cr after tNuData
      end if
   end repeat
   set the htmlText of fld "table 1" to tNuData
   sort lines of field "table 1" descending numeric by  item 13 of each
   --
   put "Elapsed time=" & the millisecs - t &" ms"
end mouseUp

This alters the original algo somewhat, but outputs the same.

It takes the plain text data from the field, and uses "repeat for each line..." to build a new variable (tNuData) into which it writes the colored text as htmlText, and then stuffs that into the field for sorting.

On my system the original script took 7621 ms; the new one takes about 48 ms.

The two links I provided in my first post here provide an explanation of how these changes delivered a performance boost of more than two orders of magnitude.

In addition to those two tips, the other two takeways here are:

1. A field's htmlText property provides a single string representing all data and attributes of a field's content. The array option is also good for some tasks, but if you can start from plain text and build htmlText from that you'll usually get good performance with simpler code.

2. Providing sample data for queries like this really helps make it easy to explore and revise the code that operates on it. Without sample data we need to guess about how the data is structured and what it contains, and the likelihood of guessing correctly is close to zero.

[user#606]

Hi Richard,
Well, this worked perfectly, right out of the proverbial box!

49ms x 4 will be well within my requirements and should not cause any noticeable issues.

I did read your links and got the point. I tried to make the change, but got an anoyiong "Repeat: missing 'in' " error. I can study this solution and be a bit wiser for it.

Thank you all for your help, if you ever visit my island Alderney in the channel islands, there is a drink or two waiting for you.

[FourthWorld]

Thank you all for your help, if you ever visit my island Alderney in the channel islands, there is a drink or two waiting for you.

I'll take you up on that, thanks. Happy to help.

[user#606]

Hi Richard,
Everything is now working as desired.
I had to change back to the line background method, because the html required

Code: Select all

set the htmlText of fld "table 1" to tNuData

At first, I thought everything was fine, but when scrolling down and working with the coloured lines above the field, the above code reset the scroll position.
This would have been ok if the polling had been manual, but with automatic polling in the background, using the table was unworkable.

Oddly, having put the line count in and directly addressing the Table 1 to put the background coloured lines, the process time is down from the excelent 49ms to just 8ms
Code as it is now

Code: Select all

 local t, tData, tNuData
put the millisecs into t
local NumLines, LineCount,Temp,Temp1, thumb
set itemdelimiter to tab
set the twelveHourTime to false
set the useSystemDate to true

put the time into Temp1
convert Temp1 to seconds
put fld "table 1" into tData
put empty into tNuData
put 0 into LineCount
repeat for each line tLine in tData
   put LineCount +1 into LineCount
       put item 13 of tLine into Temp
if Temp > 0 then
   --put "<font color=red>" & item 6 of tLine &"</font>" into item 6 of tLine

   if Temp > Temp1 then
      --put "<p bgColor=yellow>" & tLine & "</p>"&cr after tNuData
      set the backgroundColor of line LineCount of field "Table 1" to "yellow"
    
else
   --put "<p bgColor=#ffcccc>" & tLine & "</p>"&cr after tNuData
     set the backgroundColor of line LineCount of field "Table 1" to "#ffcccc"
end if
else 
put empty into item 13 of tLine
--put "<p>" & tLine &"</p>" &cr after tNuData
end if
end repeat
beep

--set the htmlText of fld "table 1" to tNuData

--priority
--sort lines of field "table 1" descending numeric by item 13 of each-- issue with scroll jump and Save Data to wrong line
put "Elapsed time=" & the millisecs - t &" ms"

--send "putTheTime" to me in 20 seconds

[jacque]

Alternately, get the scroll of the field before you run the update and reset it afterward.

put the scroll of fld "table 1" into tOldScroll
-- run the updating part
set the scroll of fld "table 1" to tOldScroll