How can an image be searched?

[SparkOut]

The binary data does indeed map the pixels in an x/y manner.

Not directly without doing some calculation.
If the image is static and the dimensions known, then you could say the y position maps to a "row" by being a multiple of the width. Then add the x value to that multiple.
Meaning that if the goal is to tell when the robot reaches the target, it's easy just to test the 5396th pixel value (or wherever you have calculated the target to be in the binary data).
It is not possible to simplify the test like this if the goal is to check where in the room the robot is as it travels.
Keeping the resolution very (very very) low would make the checks easier.

[dunbarx]

Sparkout.

I guess I just don't have any experience with imageData.

I imported a small image onto a card, and placed its imageData into a variable. The output was peculiar. I asked how many lines were in that variable, and got 9. But upon examining the variable itself, I can see 14 lines. These are not all fully visible, in that several run way off the screen. They do not seem to be wrapped. I can verify 9 lines that are relatively short.

I have no idea how to deconstruct that data into anything at all. Here is the variable:

Screen Shot 2022-11-09 at 10.24.17 AM.zip

(232.11 KiB) Downloaded 191 times

Craig

[FourthWorld]

I imported a small image onto a card, and placed its imageData into a variable. The output was peculiar. I asked how many lines were in that variable, and got 9. But upon examining the variable itself, I can see 14 lines. These are not all fully visible, in that several run way off the screen. They do not seem to be wrapped. I can verify 9 lines that are relatively short.

What is a "line" in a binary bytestream?

It's possible that some binary data will include bytes that equate to what in textual operations would be a line delimiter. But binary data is still binary data, and textual operations on it will be of little practical value.

The way to work with imageData has been provided. Try what's been suggested by those who work with it regularly, and then see what remaining questions emerge.

[jacque]

I've needed a mouseColor function for a very long time, and have collected some handlers by others over the years. I'm sorry I didn't save the name of the clever export method, but here's what I have in my library. Mine is good for a one-off but you might want to test one of the other methods for repeated calculations to see if they're faster.

Code: Select all

My version:

function getPixelColor pPt -- pass local stack-based pt
  put the screenmouseloc into tOldLoc
  put (item 1 of pPt) + (the left of this stack) into tloc
  put (item 2 of pPt) + (the top of this stack) into item 2 of tloc
  set the screenmouseloc to tloc
  put the mousecolor into tColor
  set the screenmouseloc to tOldLoc
  return tColor
end getPixelColor
------
Unknown author:

function readPixelColor pVal
   put globalLoc(pVal) into pVal
   put item 1 of pVal + 1 into item 3 of pVal
   put item 2 of pVal + 1 into item 4 of pVal
   export snapshot from rect pVal to templateImage
   get the imageData of templateImage
   reset templateImage
   return byteToNum(char 2 of it) & "," & byteToNum(char 3 of it) & "," & byteToNum(char 4 of it)
end readPixelColor

-------
Scott Rossi:

local theImageData

function getPixelColor pImage,pX,pY
   if theImageData is empty then put the imageData of pImage into theImageData
   put width of pImage into W
   put ((pY-1)*W)*4+pX*4 into lastChar
   put charToNum(char lastChar of theImageData) into B
   put charToNum(char lastChar-1 of theImageData) into G
   put charToNum(char lastChar-2 of theImageData) into R
   return R,G,B
end getPixelColor

This should retrieve the color of a pixel in a target image based on the X,Y
offset of the touch from the topLeft.  The idea is to store the imageData of
the target image in theImageData variable so the function doesn't have to
retrieve it repeatedly.

I've been wanting mouseColor on iOS myself and while not true mouseColor,
this comes close.  Hope this helps.

[dunbarx]

Richard.

With my little image on the card, I put this into a button script:

Code: Select all

on mouseUp
   put the imageData of img 1 into temp
   repeat with x = 1 to the height of img 1 step 3
      repeat with y = 1 to the width of img 1 step 3
         repeat with u = 1 to 4
            put byteToNum(char y of temp) & "," after accum
         end repeat
         delete the last char of accum
         put "*" after accum
      end repeat
      delete the last char of accum
      put return after accum
   end repeat
end mouseUp

This yields the X/Y "array" I was looking for. Each line in accum scans the pixels in the image horizontally , and the totality of lines scans vertically. It certainly is faster than the mouseColor "trick"

I assume now that It is just a matter of checking the RGB numbers in sequence, delimited by "*" to find a trio of values close enough to the target value. A little wiggle room amongst the separate RGB values should catch it. The step value of 3 in the above handler ought to resolve adequately, though this may take a little experimentation.

Craig

[dunbarx]

The reason I was confused is that I never looked closely at the actual string of chars returned by the imageData property. I just saw it as computer gibberish. These chars were mostly tiny unrecognizable "dots", with an occasional recognizable one. It was the fact that mostly these chars, at least for the few times I ever looked at them, were from areas of white or nearly white (from 250 or so to 255). Those values are exactly those very dots.

What do you know, it is just a bunch of chars who's ASCII values are the RGB values of the underlying pixels. What a clever and compact way to characterize an image.

Oh, everyone already knew that.

So why cannot a button or graphic have a similar property? The "graphicData".

Craig

[bn]

Craig,

Glad you figured out imageData.

Here is a stack that uses imageData to show the "mouseColor". The script is in the card script.
There is a mouseMove and a mouseUp handler to get the "mouseColor"
I hope I got the math right...

If you change the dimensions of the image manually then you have to initialise using the button "initialise"

ColorOfMouseLocInImage.livecode.zip

(194.76 KiB) Downloaded 180 times

Kind regards
Bernd

[dunbarx]

Bernd.

But why not just have this in the card script?

Code: Select all

on mouseMove x,y
   if the mouseLoc is within the rect of img "myImage" then set the backColor of grc "gTestColor" to the mouseColor
end mouseMove

Craig

[bn]

But why not just have this in the card script?

Code: Select all

on mouseMove x,y
   set the backColor of grc "gTestColor" to the mouseColor
end mouseMove

Because I thought this whole exercise was to work with imageData.
At the same time it is a way to make sure that the color of the pixel under the mouse derived from imageData seems reasonable.

Kind regards
Bernd

[dunbarx]

Bernd.

Ah. OK.

I have never used imageData for anything. Just not how I work in LC.

Craig

[FourthWorld]

It was mentioned that judging the distance the robot was from the camera would be hard to calculate. It isn't an issue, because the camera would be looking down, not along the robot path.

That simplifies things, as does:

Ignore scale and ambient lighting for now. Perhaps there is no room lighting, for the purpose of this application.

If the situation allows for such unusually narrow constraints, it may be doable without external libraries.

I haven't actually seen confirmation that the x,y value can be determined, at all, by the software.

True, there is no "Make My Application" button.

But the raw materials needed have been presented here.

Have you tried bringing in a sample image of the sort you expect to work with, and begun exploring options for walking thru the imageData for the color you're looking for?

Understanding how to work with imageData will be central to this exercise.

[ajperks]

Thank you all for your input.
I will be looking at the problem again on Sunday and hope to report back.

[jiml]

ajperks,

Put this in the message box.

Code: Select all

go stack url "https://netrin.on-rev.com/searchimage/pixelColor.livecode"

This stack will demonstrate how to find the location of your robot within a camera image.
It assumes the color of the robot is known, constant and unique.
As Richard points out that is a tall order!!!

But this should get you started as it is very fast.

Jim Lambert

[dunbarx]

Jimi.

Using the "byteOffset" function is a clever and fast method.

But it seems odd to me that the stack works as well as it does, in that the robot is green, and I see no such green, nothing even close, in the screenshot.

How do it know?

Craig

[jiml]

Well, that's odd. Are you sure?
The screenshot is exported from the 'camera' group.