filter array

[jameshale]

I have an indexed array of text segments.
I would like to find those elements of the array that contains a specific word.
I thought FILTER would do this.
I tested out using filter with the following code:

Code: Select all

on dosearch
   local thesearch, sctext
   --present answer dlog for search
   ask "Please enter the word you wish to search for:" titled "Search"
   put it into thesearch
   if thesearch is not empty then
      put ctext into sctext
      put "*" & thesearch & "*" into thesearch
      filter elements of sctext with thesearch
      put the keys of sctext
   end if  
end dosearch

Note I copy the array "text" to a temp array "sctext" to carry out the "search"
"ctext" is a global
Expecting to get the keys of all the elements of the array containing the search word.
stepping through the handler when I step over the filter command the array is emptied.
The array I am testing this on does have the search word in many of the elements

what am I doing wrong?

[SWEdeAndy]

I tested similar code and it worked as expected, if the array is just one dimension.
Can you post a small subset of array sctext so we can test with the exact same structure and data?

[jameshale]

“ if the array is just one dimension.”

Doh!

[jameshale]

Not to be deterred I copied the dimension I waa interested into a 1 dim array (same key) and yes the filter worked as expected.
I then wondered if tis was still faster than using a repeat and if "contains" structure to get the list of keys so I compared the two/

Code: Select all

if thesearch is not empty then
      repeat for each key tKey in ctext
         put ctext[tkey]["text"] into sctext[tkey]
      end repeat
      put "*"&thesearch&"*" into thesearch
      filter elements of sctext with thesearch
      put the keys of sctext into slist
      replace return with comma in slist
   end if
  --now use a repeat to search the array
   if thesearch is not empty then
      put empty into slist2
      repeat for each key tKey in ctext
         if ctext[tkey]["text"] contains thesearch then
            put tkey&comma after slist2
         end if
      end repeat
   end if

Result (in msecs):
array to array then filter: 4
repeat 5
not a noticeable difference but still. Filter is fast!

Note the array in question only had 425 keys (elements)
it was of the form
ctext[key]["ref"]
ctext[key]["text"]
where I wanted to search the "text" dimension

[rkriesel]

Hi, James. Here's a technique that's less work for you and for LC.

Code: Select all

on foo
   local ctext
   put "foo" into ctext[1]["ref"]
   put "bar" into ctext[2]["text"]
   put "baz" into ctext[3]["text"]
   
   filter the keys of ctext where ctext[each]["text"] contains "ba"
   
   breakpoint
end foo

Does that work for you?
-- Dick

[jameshale]

You would think so, wouldn't you.
I tried with my dataset but found directly addressing the 2nd dimension directly was 30 - 50% slower!
Three runs with my current handler 996,970, 1560 msecs
Three runs with suggested: 2701 2998, 2960 msecs

It surprised me. I guess accessing the second dim like that has more overhead than with a single dim access (even with adding on the repeat copy)

[bn]

James, Dick

I made up a little stack that filters an array of 100.000 elements using "filterOneDimensional", "filterLines" and Dicks version.

In my testing "filterLines" is the fastest, then Dick's "2nd dimension" and then "filterOneDimensional"

When using the stack you have to generate the array first by clicking button "generateArray" which is stored as a custom property of the card.

Kind regards
Bernd

[jameshale]

Brilliant.
Thanks Bernd

[stam]

Code: Select all

   filter the keys of ctext where ctext[each]["text"] contains "ba"

Just a very minor point - you don't need the array name before [each] unless you're filtering by keys of a sub-array, ie

Code: Select all

filter elements of <array> where each[<key>] is/is not/contains/does not contain/etc <some value>

works just fine

[rkriesel]

Just a very minor point - you don't need the array name before [each] unless ...

For me that is a very important point: it explains why my mis-coded filter statement took twice as long as Bernd's repeat loop. Now the two techniques' times appear very similar.
Thanks, Stam.
--Dick

[bn]

Just a very minor point - you don't need the array name before [each] unless you're filtering by keys of a sub-array, ie

Code: Select all

filter elements of <array> where each[<key>] is/is not/contains/does not contain/etc <some value>

works just fine

Thanks Stamm for this hint. I finally understood that syntax variant.

For others who find it difficult to use that syntax I offer two versions that fit the requirements of James initial problem that can be added to the sample stack above.

Variant 1 is "destructive" for the original array "cText"

Code: Select all

on mouseUp
   local cText, thesearch, t1, t2, tKeys
   put the myArray of this card into cText
   put "apple" into thesearch
   put the milliseconds into t1
   
   filter elements of cText where each["text"] contains thesearch
   
   put the keys of cText into tKeys
   replace return with comma in tKeys
   put the milliseconds - t1 into t2
   put t2 & " ms" && the number of items of tKeys && "numItems" && the long time into field 1
end mouseUp

Variant 2 uses "filter into" a second array "cText2" and keeps the original array "cText" intact with almost no time overhead

Code: Select all

on mouseUp
   local cText, thesearch, t1, t2, tKeys, cText2
   put the myArray of this card into cText
   put "apple" into thesearch
   put the milliseconds into t1
   
   filter elements of cText where each["text"] contains thesearch into cText2
   
   put the keys of cText2 into tKeys
   replace return with comma in tKeys
   put the milliseconds - t1 into t2
   put t2 & " ms" && the number of items of tKeys && "numItems" && the long time into field 1
end mouseUp

I hope I got the syntax right.

Kind regards
Bernd