Arrays

[gjn1w07]

Hi all,

I'm trying to write a program that simulates a theory of memory and I've run into some problems. Up till now I've been able to get by with repeat loops and if statements, but the magnitude of the data I'm dealing with means that one run of my simulation would take 70 hours or so, which is way too long for my purposes. I think I need to use arrays!

There are two problems I have, so I'll split these into two sections

1) Creating the arrays.

At the moment I have a variable that creates a number of lines populated with 1s and -1s. So imagine I want to have 3 lines that look like this:

1, -1, 1, 1
1, 1, -1, 1
1, -1, 1, 1

At the moment I simply define the parameters for the number of lines and the number of items in each line, then generate the numbers randomly. So in this case I've got 3 lines with 4 items in each. Having randomly generated these 3 lines, I then append a non-random sequence of numbers onto the end, lets say -1, -1 in this case. So I end up with 3 lines with 6 items each. How do I convert this to an array? I tried to use

Code: Select all

split gNumberVar by comma and return

but because all the entries are either 1 or -1, this didn't work.

2) Analysing the arrays

In my simulation I will have two arrays - lets call them the memory array and the word array. The memory array will in practice be quite big - around 1000-2000 lines, whilst the word array will be around 120 lines. Each line in the memory array will be a series of 1s, 0s and -1s. So if the memory array was 5 lines it might look like:

1, 0, 1, 1, 0, -1
1, 1, 0, 1, -1, -1
1, -1, 0, 1, -1, 0
1, -1, 0, 1, -1, 0
1, -1, 0, 1, -1, 0

The word array will consist of 1s, and -1s and each line will be of the same length as the lines in the memory array (so if each line in the memory array has 6 items as here, then so will each line in the word array). What I need to do is the following steps:

i) Take the first line of the word array
ii) Take the first line of the memory array
iii) Multiply each item in the word line sequence by it's companion item in memory line sequence. So if the first line of the word array is 1, 1, 1 and the first line of the memory array is -1, 0, 1 then I want to do three multiplications: 1*-1, 1*0 and 1*1.
iv) Add up these values (lets call this tMatches)
v) Divide tMatches by the number of non-zero entries in the memory line sequence
vi) Add this number to a running total (lets call this tIntensity)
vii) Take the second line of the memory array, perform the same process with the first line of the word array, and then add this to the tIntensity running value
viii) Once the first line of the word array has been compared to all lines of the memory array, the current value of tIntensity is written to a file (I can do this bit!) and tIntensity is reset
ix) Take the second line of the word array and the first line of the memory array and do the whole process again

So in other words the first line of the word array will be compared to all lines of the memory array, then the second line of the word array will be compared to all lines of the memory array and so on until I have a tIntensity value for each line in the word array.

Any help on this would be very much appreciated. I've tried programming this using repeat loops, and predictably the whole thing is just way too slow due to the size of the memory array. I've tried looking in the help files and tutorials about arrays, but I can't find what I'm looking for there. Even a prod towards the right things to read would be appreciated - I've read everything I can find in the manual and online tutorials about arrays and I can't even figure out how to make the array in the first place let alone start on the array comparisons!

Sorry for the long post.

[Mark]

gjn1w07,

First a quick word on arrays. People often tend to think that arrays are matrices, but this is wrong. Arrays are linear and have the following form:

(1,1): 1
(1,2): 2
(1,3): 3
(2,1): 4
(2,2): 5
etc.

In Revolution, you can assign elements to arrays as follows (this is just a simple example):

Code: Select all

put 0 into myValue
repeat with x = 1 to 10
  repeat with y = 1 to 10
    add 1 to myValue
    put myValue into myArray[x][y]
  end repeat
end repeat

Now you'd have a "two-dimensional" array corresponding to a matrix with 10 lines and 10 columns and values 1 to 100. The array itself, though, it linear. You can use a similar script to create an array from your own matrix. You will probably need something like:

put item x of line y of myMatrix into myArray[x][y]

Once you have created the array, you can do calculations on it. Example:

Code: Select all

repeat for each key x in myArray
  repeat for each key y in myArray[x]
    repeat for each key p in myOtherArray
      repeat for each key q in myOtherArray[p]
        put myArray[x][y]*myOtherArray[p][q] into myTempVar
        -- do something with myTempVar here
      end repeat
    end repeat
  end repeat
end repeat

If this still takes too long, you might want to add a progress bar and modify the script to update the progress bar. You'd have to modify the repeat loops by adding "with mesages" and "wait with messages":

Code: Select all

repeat for each key x in myArray with messages
  -- do something
  wait 0 millisecs with messages
end repeat

I hope this helps.

Kind regards,

Mark

[gjn1w07]

Oh thank you so much - that's exactly what I needed. It's going to take me a while to update my program to work in the way you suggest, but I think that will help a lot. You make it all sound so simple!

I have a related question that should get me to the point where I can run the simulation and see if it doesn't take 70 hours. I've modified your function so that it looks a bit like this:

Code: Select all

   put "" into gAbstractFeatures
   put "Hello" into gAbstractFeatures
   put comma after gAbstractFeatures
   put "There" & comma after gAbstractFeatures
   put "World" after gAbstractFeatures
   put "" into myArray
   put 10 into gStimNumber
   put 4 into gStimLength
   repeat with y = 1 to gStimNumber
      repeat with x = 1 to gStimlength
         put random(2) into tNumberCheck
         if tNumberCheck is 1 then put -1 into myArray[x][y] else put 1 into myArray[x][y]
      end repeat
      put 1 into tAbstractItem
      repeat for each item thisitem in gAbstractFeatures
         put thisitem into myArray[(gStimlength + tAbstractitem)][y]
         add 1 to tAbstractItem
      end repeat
   end repeat

So essentially it's populating an array with size x=gStimLength and y=gStimNumber with random numbers of 1 and -1. After that it is going down each line of the array and appending "Hello there world" after each line of the array.

Now I want to be able to examine the size of the array, because later on I'll be doing things like appending more lines onto the bottom or the side. I've seen that there is a command "Extents" which sounds like it should give me what I want. However, the following code simply returns 1,7:

Code: Select all

put the extents of myArray into myVar
Answer myVar

I also tried:

Code: Select all

put item 2 of line 2 of the extents of myArray into myVar
Answer myVar

Which returns blank. Since I've got a 2 dimensional matrix, shouldn't extents be returning something like 1,7 and 1,10? Sorry if these are obvious questions but I'm pretty new at using arrays!

Also, I can't seem to get it to display the whole array in a field. I know that you have to use a command such as:

Code: Select all

combine myArray by return and tab

But if I do this, I dont see the whole array. The combine code above gives me 1,2,3,4,5,6,7 which I presume are the keys of the x dimension. Now I know that the elements are in the array because a command like "answer myArray[x][y]" gives me the answer I would expect (I.e. answer [5][1] gives me "hello").

The extents question I actually need for my coding, the second is more so I can properly understand how arrays work.

Thanks again for the answer already given - you've already been awesome.

[Janschenkel]

The extents function comes from the days before the engine started supporting nested arrays. There is nothing wrong with using the older way of comma-separating the indices to achieve a 'matrix' - just remember that in revTalk, arrays are actually 'dictionaries' or 'maps', and the index that you pass is used to find the element.
In other words, when you use

Code: Select all

put MyArray[1,4]

what actually happens is that the engine concatenates the indices into a single string "1,4" and then looks up the corresponding entry in its map. It's actually pretty smart about getting rid of whitespace in the indices, so

Code: Select all

put MyArrau[1  ,  4]

will still get looked up as "1,4".

HTH,

Jan Schenkel.

[bn]

Hi gjn1w07,

(it feels really easy to pronounce your name )

If I understand what you described in your first post correctly I think you are better off without arrays. You really don't have any speed advantage, in my opinion. Arrays are a beautiful thing and I strongly support them but I dont see where you need them.
I gave it a shot without arrays but with a bit of the "repeat for each" element form, which is very very fast in runRev. Have a look at it.

If I did not goof up on the implementation of your first post then for 120 wordLines and 2000 memoryLines with 6 random items in each line, it takes 1400 milliseconds to do the calculation. Which is a lot shorter then the 70 hours you mentioned, so it might well be that I got it wrong but please look at the appended stack.
It lets you create 2000 memory lines and 120 word lines and then lets you calculate the intensity per word line against all the memory lines.
I commented the stack.

Feel free to ask or correct me if I got the algorithm all wrong.

regards

Bernd

[gjn1w07]

You got it spot on! Thanks very very much for this. I've had a look at your code and compared it to what I had already. Not only have you given me exactly the code I needed, but I can now see where in my code things are slowing down. At least I think I can assuming that both "repeat for each" and "repeat with" are much faster than "repeat until". And not only is the code faster, but I dont have to spend my weekend coding the thing

Thanks very much to everyone that replied to my post - I know tonnes more about arrays and what is fast and slow in revolution than I did before.

Sorry about my user name by the way - I dont think I was really thinking straight when I picked it

[bn]

Hi gjn1w07,

glad I got the algorithm right.

The thing is that "repeat for each" is a lot faster then "repeat with" because in long lists repeat with has to do the counting everytime around whereas repeat for each remembers the last position of e.g. a line in a list of lines. That is the reason why I have put the counter for the wordLines in since you dont know which line is processed as you would with the repeat with i = 1 to x.

Another speed consideration is to put everything into varibles, no field access and especially no disk accesss, these slow things down.
Yet another speed trick is to append to a variable (put myVariable & return after myCollectorVariable) instead of "put before" or "into line something".

Since your data is well structured and you want to process it line by line and it is just two nested lists of lines and than some operation on the content of the lines, you do not gain much from the array approach. Arrays are fantastic if you need non-consecutive access to your data. In your case they just make things more complicated, and probably a bit slower.

good luck with your project and a nice "coding-free" weekend

regards
Bernd