How to optimize image processing and rendering in LiveCode - Emmanuel Katto Uganda

[emmanuelkatto]

Hey everyone, I am Emmanuel Katto. I'm working on a LiveCode project that involves heavy image processing and rendering, including resizing, cropping, and applying filters to high-resolution images. I'm experiencing slow performance and memory usage issues. Are there any optimization techniques or best practices I can use to improve the speed and efficiency of image processing in LiveCode? Can I take advantage of multi-core processing, GPU acceleration, or other features to speed up my app? Please let me know.

Thanks in advance!
Emmanuel Katto

[stam]

Dear Emmanuel, without being sure, I'm going to guess that no, you can't use GPU acceleration specifically. Also, LC runs in a single thread, so by default, you will not be able to do multi-core processing.

Having said that often in this forum people present slow code and often it can be sped up by orders of magnitude just using more apt techniques (handlers that take 2-3 minutes end up taking 10 milliseconds etc).

I would suggest maybe you post your slow code here - hopefully some will be able to help.

Regards
Stam

[mtalluto]

You can spin up multiple copies of your app and have them process different images in parallel. This works well with modern CPUs that have multiple cores.

Our CRM will spin up headless processes, tell them what to work on, and quit them when they are done.

[stam]

Thanks Mark - that's interesting and a bit like the XOJO 'worker machines'. I'm guessing this must be on Windows though as by default on MacOS, you generally can't run multiple instances of the same app... or am I wrong?

[richmond62]

As far as I am aware you cannot run multiple instances of an app with the same name in MacOS. But there is nothing to stop you hiving off multiple standalones with different names: myStuff1, myStuff2, myStuff3 . . . which you should be able to run simultaneously on a Mac.

[FourthWorld]

I don't know of a way to launch multiple instances of an app from the GUI, but macOS is a certified Unix so there's always a way to do just about anything from shell():

Code: Select all

open -n -a APPLICATION-NAME

https://stackoverflow.com/questions/473 ... ion-in-mac

Useful as multiprocessing is in some circumstance, it's not a universal panacea for general performance. Breaking an unusually large task into smaller parts running in parallel carries its own coordination overhead, sometimes more trouble than it's worth.

And none of the instances can run any faster than the current code, so let's first optimize the code before we decide to make it more complex

--

The OP states the following needs:

- resizing
- cropping
- applying filters

The good news is two of those three (resizing, cropping) are handled in the engine via compiled object code. Those can be done very quickly with the LC engine.

The bad news is that while it's possible to write image filters in a scripting language, it's rarely done. Even where the big players like Adobe will use a scripting language for the GUI, as they did with Lua for Lightroom, they use optimized compiled code for filtering, usually written in C++ or C.

BUT -

We don't yet know what those filters are, and LC has a LOT of flexible options for tinting, blurring, and more.

So first, Emmanuel, could you describe what those filters need to do?

This may be very easy. Or it may be very hard. It will depend on knowing what's needed.

--

While we wait for Emmanuel's reply, for the sake of completeness I'll outline options for image filtering here in case we need them for this, or perhaps someone else might need them down the road.

There are at last three ways to approach this:

A) External: Do what most do and write computationally-intensive components in C. LC provides an externals interface for that purpose.

There's enough sample code in the world for image processing in C that I suspect the custom coding will be mostly wrapping it in LC's externals interface.


B) Dedicated App: Find some way to offload image filtering to an external process optimized for that.

Thomas McGrath once shared some sample code here for using Automated to hand off filtering to Quartz Composer, which may inspire a more modern solution.
viewtopic.php?f=19&t=1653


C) Bite the bullet and script it.. This will be slow, and a lot of work. But if nothing else works out, as the great poet Donald Rumsfeld used to say, you go to war with the army you have.

Back in 2012 there was a thread here on image manipulation, where Bernd contributed a handler for brightening an image pixel by pixel, which may be a useful starting point for other filtering:
viewtopic.php?f=10&t=9490&start=75#p58787

This thread from the year before includes code examples for pattern stamping and gradients (the latter is now well handled in LC directly, but a good example of pixel operations):
viewtopic.php?f=6&t=8918&start=15

Somewhere in this community there was once even a convolver, but as you can imagine it was veeeeeery slow relative to what people are used to from compiled apps that I'd did into my archives to see if I can find it only if no other option proves useful.

--

Let's find out what Emmanuel needs, and hopefully it's something that can be done gracefully and quickly without pixel-by-pixel operations.

[stam]

Well, hopefully at some point compiled scripts will materialise (not holding my breath tho lol) and stuff like this will be sped up greatly…

[mtalluto]

Here is an interesting story related to what we used to call Minions.

We called them minions to reference the imaginary henchman who did all the work for their leader. This was long before the movie of the same name was conceived.

Years later, we demoed the processing speed provided by this technique at a LiveCode conference in Edinburgh many years ago. The description of our talk mentioned the Minions.

The demo that showed what they could do went well with the audience. At the very end, a guy in a suit came up to congratulate me on our work and handed me his card with a smile. It was a Universal attorney. He said nothing more, and we promptly renamed them Boosts.

[mtalluto]

Thanks Mark - that's interesting and a bit like the XOJO 'worker machines'. I'm guessing this must be on Windows though as by default on MacOS, you generally can't run multiple instances of the same app... or am I wrong?

The solution is to have a small executable that loads libraries that do all the work. You rename the executable and load it. Rinse and repeat for the number you need. I found that ten at the time was a maximum as with 5 or so being a happy spot.

I started to API them for general use but never finished.I wanted to make the available as an added feature for LiveCloud users. Imagine being able to do a big sync that did not stop your application, as one example. I'll dig up the start up code.

We had them working on Mac, Win, and Linux. I wonder if the video LiveCode did on the session is available somewhere. The demo included our CRM processing a thousands of network tasks. I started the system processing without them. As it was working I turned them on. There was a countdown field and it went ballistic when the minions were engaged. We received an audible appreciation.

I think this form of sharing a workload is applicable to many things. It will not solve everyone's needs.

I remember HH demoed some cool image manipulation. His work was impressive. I think he was doing some JS in a browser widget. Using the browser may be another solution for the OP as code executes well in there. There are free libraries available for all things image manipulation.

[richmond62]

Let's find out what Emmanuel needs

Let's hope Emmanuel returns.

[Zax]

A) External: Do what most do and write computationally-intensive components in C. LC provides an externals interface for that purpose.

There's enough sample code in the world for image processing in C that I suspect the custom coding will be mostly wrapping it in LC's externals interface.

And D) : call Python from LC. This is what I use for some of my personal standalones. I find that writing a Python script is less complicated than writing an external, but I suppose it depends on one's technical knowledge.

[stam]

That’s a good call actually. And an excuse to learn/refresh Python

[Zax]

Concerning image processing, maybe it's possible to install Image Magick and call it with its Command-line Tools from LC.
I didn't try.

[SparkOut]

Before he died, Hermann (forum name [-hh] ) used to do some *amazing* things with image processing, often leveraging ImageMagick and javascript, and rendering in a browser widget. (Among coding stacks for many other areas of use too.)
Sadly a lot of his contributions were taken down, but there might be some still available if you search here.

I miss Hermann.

[PaulDaMacMan]

Before he died, Hermann (forum name [-hh] ) used to do some *amazing* things with image processing, often leveraging ImageMagick and javascript, and rendering in a browser widget. (Among coding stacks for many other areas of use too.)
Sadly a lot of his contributions were taken down, but there might be some still available if you search here.

I miss Hermann.

I have saved almost all of Hermann's great work including his web-deployed stacks, almost all of it was open-source / MIT license. Really good stuff!

Concerning image processing, maybe it's possible to install Image Magick and call it with its Command-line Tools from LC.
I didn't try.

You certainly can, and ImageMagick has it's own built-in scripting language too, so you can pass in complex image manipulating scripts via shell() or open-process.