But how is this possible?

[NoN']

Like many others here, I've experienced slowness in using recent versions of Livecode compared with older versions of the software, particularly under Windows.
I've read explanations on this subject in the Livecode forums, particularly those concerning the new calculation algorithms, which integrate verification procedures that didn't exist before. So be it.

But I hadn't yet quantified this slowdown in any serious way.

To make an administrative job easier (a "chore" to be honest), I've created a little tool that lets you perform simple manipulations on text (see attachment).
In a nutshell, this tool extracts all the words in a text, excludes some of them, and calculates the occurrences for each of the words retained after extraction. Simple in principle and programming!

As I found Livecode 10's response very long in coming, I went back to version 4.5 of the software to compare response times.
And then my arms fell off!

The subjet:
a text, in English, of 7537 words and 55652 characters.

The conditions:
PC running windows 10
Processeur Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz 2.50 GHz
Mémoire RAM installée 32.0 Go (31.7 Go utilisable)
Stockage 466 GB HDD TOSHIBA MQ01ACF050
Carte graphique NVIDIA Quadro K610M (996 MB), Intel(R) HD Graphics 4600 (113 MB)
Type du système Système d’exploitation 64 bits, processeur x64

The results:
Livecode 10, extraction completed in 244 seconds
Livecode 9.05, extraction completed in 289 seconds
Livecode 4.5, extraction completed in 10 seconds

Needless to say, I re-run the script several times to ensure that the results were solid, and that the hardware conditions were exactly the same...
Running the tool under Mac (Mac mini M2, Os 15) and Livecode version 10 didn't bring any significant acceleration compared to running it under Intel/Windows.

So while I can understand a simple to double, or even triple, difference, the fact that it takes 24 times longer to run the same script sequence astounds me.
At that point, you'd expect LC 10 results to be more accurate than LC 4.5, but a test with french text (24207 car. for 3611 word) shows that this isn't the case. On the contrary, the result obtained with LC 4.5 in 2 seconds is more accurate than that obtained with LC 10 in... 61 seconds!

The icing on the cake: a routine for excluding the number of spaces in the english text (see stack script) runs in 1 tick on LC 4.5 versus 3445 ticks on LC 10.
Yes, you read that right, 3445 times longer!

What could possibly justify such a step backwards?

[Klaus]

Could you please upload that stack again with your text in fld 1?
Thanks!

[bn]

Hi NoN',

Could you try this replacement code for your repeat loop?

Code: Select all

   repeat for each word aWord in ron
      add 1 to comb
      set the thumbPosition of scrollbar "quantitmot" to round((comb*100)/nbWd)
      if char 1 of aWord is in ledeb then delete char 1 of aWord
      if last char of aWord is in ledeb then delete last char of aWord
      if (char 1 to 2 of aWord) is in aExcl then delete char 1 to 2 of aWord
      if aWord is a number or aWord is "" then next repeat
      if hilite of btn "SupPrep" and aWord is in prepCour then next repeat
      if hilite of btn "SupCoord" and aWord is in conjCoord then next repeat
      if hilite of btn "SupPron" and aWord is in pronPers then next repeat
      if hilite of btn "SupArt" and aWord is in supArt then next repeat
      put aWord & tab & 1 & return after ronin
   end repeat

I moved the conditional out of the repeat loop and placed it above that repeat loop.

Does it all you wanted from the repeat loop?

Kind regards
Bernd

[NoN']

@ Klaus
With great pleasure.

If I didn't put the text used, it's because I consider that what has been observed should remain true with any text used (at least in this format)...
But here's the stack with the text in English. I can't put the text in French because it's a personal, administrative document, as indicated in my initial post.

[NoN']

@ Bernd

Hi Bernd,
Knowing you, I'm quite sure your code will be much faster than mine.
But I won't give it a try just yet, because the important thing here is not to improve the quality of the script, but to understand why there is such a discrepancy in the execution of the same script between a fairly old version of LC and the latest version of the same software.

Your code will just run in 5 seconds under LC 4.5 and 350 seconds under LC 10 (I'll do the try, of course), but for now, what I want to understand is : how can you go from 1 tick to more than 600 times the same time to do exactly the same thing with exactly the same computer equipment?

[bn]

how can you go from 1 tick to more than 600 times the same time to do exactly the same thing with exactly the same computer equipment?

I think the main problem is that LC >= 7 uses unicode. And you access lines and words in your repeat loop quite often. Every time LC has to count lines and words which is much more involved using unicode.

Kind regards
Bernd

[dunbarx]

All.

Like Bernd, I immediately saw that using "repeat for" instead of "repeat with" might just save the day all on its own. But the last loop, at least as written, requires an index, since it makes continual reference to the preceding line in each pass. "Repeat for" does not support this. "Repeat with" does, which is why it is so often very useful, if inefficient.

I never thought of it, but here is a sample stack that kluges a way to find successive duplicates in 20,000 lines of a field using the "repeat for" structure. It is not as fast as using "repeat with". I am wondering why, since there is a single "logic" step in each. Or is there? Is that the culprit?

But given the concept, is there a way to make this work better?

Craig

Repeat forwith comparison.livecode.zip

(14.53 KiB) Downloaded 104 times

[dunbarx]

Forgetting kluges for a moment, this is the first time I have seen a compelling reason to keep older versions of LC handy. New features are always welcome, though it has been a long time since any one of these has made me really happy (the multi-char itemDelimter).

But this is another one, possibly: SPEED. Real speed.

Are we missing something? Is unicode here at a tremendous cost?

Craig

[bn]

Hi Craig,

this code is functionally equivalent to your "for each code"

Code: Select all

on mouseUp
   put "" into fld 2 ; wait 20
   get fld 1
   put 0 into lineNum
   set the cursor to watch
   put the ticks into startTime
   
   put empty into tPrevLine
   repeat for each line aLine in it
      add 1 to linenum
      if aLine = tPrevLine then
         put linenum & space & aLine & cr after accum
      end if
      put aLine into tPrevLine
   end repeat
   
   put accum into fld 2
   answer the ticks - startTime && "ticks"
end mouseUp

Kind regards
Bernd

[dunbarx]

Bernd.

Yes, just a different take on it. But did you time it? I was surprised at my 7:1 difference in speed difference (59 ticks for "with" versus 359 ticks for "for") between the two different repeat types. The very similar shenanigans that go on within each loop seem like they should each contribute very little to the overall process.

Craig

[dunbarx]

Bernd.

After making sure that the burden on each repeat type was identical, the "repeat for" is, not a surprise, much faster.

Craig

[NoN']

Dear Craig, dear Bernd,

Thank you for your answers and suggestions for improving the initial script.
I'm sure I'll gain a lot from following your advice the next time I program looping code.

But, at the risk of repeating myself, that's not the problem!

Because even if my script can be improved, if it executes in 1/2 second I'll never notice, provided, of course, that the result is right.

But in this case, not only does execution take longer, but the result is less precise, not to say false, when using a language like French. What's important here is the speed of execution relative to the version used, which varies in aberrant and counter-intuitive proportions (at least with regard to the evolution of computing).

Have you tested the improvements you propose for LC 4.5 and LC 10, respectively?

I was able to check with a text editor that my French text contained the word “savoir” 6 times (it's not even a word with an accent or a special character that could fool an algorithm). LC 4.5 correctly calculated the 6 occurrences of this word, while LC 10 calculated 5 occurrences for this term. So not only is there a slowdown, there's also a potential bug.

Is unicode here at a tremendous cost?

Craig's question is important, really important. I use Livecode in particular for its ability to process fairly large quantities of data at speeds that very quickly make the time spent writing the code worthwhile and, above all, get results without generating impatience. If I observe a multiplication of processing time by : 10, 200, 2500 or 4250 times, I'm going to start looking very seriously at other programming languages.

And I don't want to...

[Klaus]

Hi Non',

I had a little time and "pimped" your "Décompter espaces" button script with the english text you provided.
Again we see the speed of the optimized "repeat for each ..." loop.
Number of chars: 48190

This script:

Code: Select all

on mouseUp
   put the millisecs into chrono
   
   ## Not neccessary, see the time the script took :-)
   ## set cursor to watch
   put fld 1 into ron
   put "0" into NbReturn
   repeat for each char tChar in ron
      if tChar= " " then
         add 1 to NbReturn
      end if
   end repeat
   put fld "NBChars"- NbReturn into  fld "NBCharsSpace"
   put the millisecs-chrono
end mouseUp

took 25 millisecs.
When i have more time i will check the other script, too, and see if we can make if faster.

Best

Klaus

[NoN']

This script:[...] took 25 millisecs.

Oh, and 25 millisecs are better than 1 millisec ?
You think it is natural for a 14-year-old program to be faster than a recent one or to see Carl Lewis faster than Usain Bolt in the same race ?

25 millisecs vs 1 millisec (and even less, sometimes) it is 24 milliseconds too long...

I imagine these results in a different context, and the slogans that would have to be adopted to sell them.

"Dear friends, good news: our video editing software is now 25 times slower than its predecessor, guaranteeing you long, quiet evenings by the fireplace.
And there's more good news: our 3D model rendering is 100 times slower, thanks to our new algorithms, and uses much more energy. From now on, your computer can also be used as a fireplace!"

Renaud

[Klaus]

Oops, sorry, completely overlooked your timing result for this button. :-/