Must be a faster way

[adventuresofgreg]

Hi there: I'm parsing a large html file and replacing placeholders marked as text strings with different text content, and the way I'm doing it seems to be taking a very long time. It takes 2 seconds on my MacPro, but 19 seconds on my Linux container. Can you think of a more efficient way of doing this?

This is how I'm currently doing this:

Code: Select all

replace "##PNPL" WITH field "TWSprof" in dashboardhtml
   replace "##PL" WITH field "accountPL" in dashboardhtml
   replace "##CCTNAME" WITH field "anonName" in dashboardhtml
   replace "##CLSDTRDS" WITH closedpos in dashboardhtml

... etc...

It's a very convenient way of customizing an html template page, but I think the replace string search is the part that is very inefficient.

[capellan]

Could you post the content of each field in different variables?
It's faster to work with data in variables, for example:

put field "TWSprof" into tData
replace "##PNPL" WITH tData in dashboardhtml

[dunbarx]

What Capellan said.

This:

Code: Select all

on mouseUp
   put the ticks into temp
   put "xyz" into var
   repeat 1000000
      replace "xyz" with "aaa" in var -- twice as fast
      replace "xyz" with field 1 in var
   end repeat
   answer the ticks - temp
end mouseUp

Craig Newman

[Thierry]

I'm parsing a large html file and replacing placeholders marked as text strings with different text content
... etc...

Hi,

Could you be more precise about the size of your html file ,
the number of different placeholders and the average frequency of your placeholders?

[jacque]

This is what the "merge" command is for. It will do all the replacements at once.

[Thierry]

This is what the "merge" command is for. It will do all the replacements at once.

I was thinking of merge but I have some benchmark where merge is
200 times slower than an equivalent list of replace, thus my questions before....

Regards,

Thierry

[adventuresofgreg]

Thanks. I think the most important bit of data that should be in a var is the main html file that contains all of the replacement placeholders (from my code below, "dashboardhtml"), which is quite large, and that is already in a variable. I could try to put all of the small bits of data into variables first, but I just don't see that being any faster (you have to place them into the var first anyhow and that means moving the data from the field into the variable.

I'm lookup merge and give that a try.

Code: Select all

put VERYlargeHTMLfile into dashboardhtml
replace "##PNPL" WITH field "TWSprof" & return in dashboardhtml
   replace "##PL" WITH field "accountPL" in dashboardhtml
   replace "##CCTNAME" WITH field "anonName" in dashboardhtml
   replace "##CLSDTRDS" WITH closedpos in dashboardhtml
   replace "##CCUNTVLU" WITH field "myvalue" in dashboardhtml
   replace "##CCUNTTYP" WITH Atype in dashboardhtml
   replace "##MS" WITH the label of button "multisysmenu" in dashboardhtml
   replace "##SYMBLS" WITH loadedsymbs in dashboardhtml
   replace "##FLTRS" WITH "no filters" in dashboardhtml
   replace "##PS" WITH positions in dashboardhtml
   replace "##CPTL" WITH field "capital" in dashboardhtml
   replace "##RSK" WITH field "riskperbar" in dashboardhtml
   replace "##indexembed" WITH theINDEX in dashboardhtml

[jacque]

I'd be curious to know what you find out. Using replace, the engine has to parse the long html file 12 times. Using merge, I think it does it all in one pass.

[capellan]

Probably I need to confirm this, but as far as I know
everytime that you access data inside a field, then
Livecode engine sends messages... but this could have
changed in recent Livecode versions.

Richard Gaskin have an utility that could confirm this:
4w Props or 4w umbrella.

[AxWald]

Hi,

dunno if it helps here, but a while ago I did some timings & optimizations with replace loops. Got some very nice speed improvements for large files. To be found here.

Have fun!

[adventuresofgreg]

Merge is about 3 times faster! THANKS!