But how is this possible?

Want to talk about something that isn't covered by another category?

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

NoN'
Posts: 96
Joined: Thu Jul 03, 2008 9:56 pm
Contact:

Re: But how is this possible?

Post by NoN' » Fri Jun 20, 2025 8:38 am

bn wrote:
Wed Jun 18, 2025 1:23 pm
Could you please test this version and tell us the timings.
Hi Bernd,

I'll test this as soon as possible and let you know the results in the meantime.

Kind regards.
Renaud

bn
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 4163
Joined: Sun Jan 07, 2007 9:12 pm

Re: But how is this possible?

Post by bn » Fri Jun 20, 2025 9:15 am

@Richmond,
Please note that it is slightly ironic that in admonishing a spelling error you committed one yourself.
En Anglais vous ecrivez le mot 'exemple' avec un 'A': exAmple.
Kind regards
Bernd

richmond62
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 10076
Joined: Fri Feb 19, 2010 10:17 am

Re: But how is this possible?

Post by richmond62 » Fri Jun 20, 2025 11:09 am

I am an ironic person: and my French is awful (something I freely admit).

J'ai un amour de bou. :lol:

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1232
Joined: Thu Apr 11, 2013 11:27 am

Re: But how is this possible?

Post by LCMark » Tue Jun 24, 2025 5:54 am

NoN' wrote:
Wed Jun 18, 2025 11:09 am
But I still say that all this takes us away from the original question, which has not really been addressed.
As a reminder, my post wasn't aimed at improving script quality (it is not the good forum for that), but at understanding the reasons behind the difference in speed between different versions of Livecode. Unicode was mentioned as one of the probable causes of this problem. But is it the only one?
It is entirely down to Unicode :)

The text in the test stack you uploaded ('Linking farmers to markets...') will be Unicode text in 10.0 as it contains several instances of unicode-only dashes. In contrast, in 4.5.3 it will be native encoded (I suspect some sort of compatibility conversion will have been done by the clipboard turning any exotic hypen into '-').

Indeed, if I take the stack you uploaded and ensure it is all native text then I get the following timings for different versions of LC I happened to (still) have installed in a Windows VM on my (now somewhat ancient) Intel iMac Pro:

Code: Select all

   4.5.3: 84 seconds
   9.6.10: 54 seconds
   Create 1.0.0-dp-5 (engine version 11.0.0-dp-5): 43 seconds
(I didn't actually have a version of 10.0 installed - but its timings will either be similar to 9.6.10, or similar to Create 1.0.0-dp-5, depending on when any internal optimizations which had an effect on this code were done).

Upshot - at least since 9.6.10 (probably earlier) - if your script is actually doing *exactly the same* as in 4.5.3 (i.e. operating on native encoded text) then the engine is now about 25% faster - if not 50% faster (in even later engines).

Unicode makes the internal text processing a lot more computationally expensive as things which were essentially indexed array lookups, and byte comparisons are no longer that. For example, 'char X of S' in 4.5.3 is essentially 'S[X]', but in a Unicode string 'char' chunks are multi-codeunit potentially (they map to graphemes). For example, unicode has two representations of é - e-acute, and e,combining-acute - both these representations are a single character and moreover must be seen as equal.

Anyway, it looks like @bn already has you covered for optimizing your scripts to make them work substantially faster and I suspect faster than they were in 4.5.3 :)

By the way, I did not see any difference between the results in the three versions of the engine I tried the English text in so I wonder if your assertion that '10 seems less accurate than 4.5.3' is actually human error (or the result of the text being converted to native encoding in 4.5.3, but retaining full unicode-ness in 10)?

NoN'
Posts: 96
Joined: Thu Jul 03, 2008 9:56 pm
Contact:

Re: But how is this possible?

Post by NoN' » Wed Jun 25, 2025 2:21 pm

@LCMark

Thank you very much for taking the time to read me and answer my initial question.
By the way, I did not see any difference between the results in the three versions of the engine I tried the English text in so I wonder if your assertion that '10 seems less accurate than 4.5.3' is actually human error (or the result of the text being converted to native encoding in 4.5.3, but retaining full unicode-ness in 10)?
Human error can never be ruled out, but the difference in accuracy was observed using a French text rather than an English one. I'll be running further tests to try to confirm or refute this observation.

@ to all contributors to this post

I certainly don't regret having asked this question, both for the script improvements suggested by the eminent members of our community and for a better understanding of the mechanisms that animate LiveCode.

As I work more and more with LC 10 and less and less with LC 4.5, these aspects are becoming increasingly important and I can see that I won't be able to avoid rewriting my old codes to adapt them to the new standards.

But this time, at least, I won't do it feeling like a waste of time.

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1232
Joined: Thu Apr 11, 2013 11:27 am

Re: But how is this possible?

Post by LCMark » Wed Jun 25, 2025 3:38 pm

NoN' wrote:
Wed Jun 25, 2025 2:21 pm
Human error can never be ruled out, but the difference in accuracy was observed using a French text rather than an English one. I'll be running further tests to try to confirm or refute this observation.
So it *might* be related to Unicode in that there might be more characters you now need to strip from either side (i.e. ledeb) - for example there's only one dash like char in native encodings ('-' - ASCII dash); but Unicode has a veritable zoo of dashes (https://www.compart.com/en/unicode/category/Pd). [ Unicode tends to classify characters semantically and *not* visually ].

If you email me the French text you see the difference between 4.5.3 and 10.0 to support@livecode.com, then I can take a look to see if there is any strange functional difference (not related to Unicode) [ If you do this, it would be best to send two files - one which is the text copied out of the field in 4.5.3, and one which is the text copied out of the field in 10.0 to make sure I can run things exactly as you do ].

NoN'
Posts: 96
Joined: Thu Jul 03, 2008 9:56 pm
Contact:

Re: But how is this possible?

Post by NoN' » Wed Jun 25, 2025 4:49 pm

@LCMark

Indeed, that's probably where the difference lies.

Thanks for your proposal, but unfortunately I can't transmit this text, otherwise I'd have done it already.
I'll try again with a more "neutral" text and keep you informed of the results...

Thanks again for your help

Post Reply