Getting tricky with iphonePlaySoundOnChannel

[richardmac]

I'm playing around with building a music app that plays 4 tracks of audio (drums, bass, piano, guitar) at the same time using 4 channels of iphonePlaySoundOnChannel. Getting all 4 tracks to play at the same time is easy. However, when you use 4 lines of code, one to call each of the 4 channels, by the fourth channel the audio is off from the original by about a 20th of a second, which in music is no good.

So I've come up with a strategy, but if anyone else has any better ideas I'd love to hear them. First I'm going to figure out how much delay is added to each sound channel I had. Let's say I find out it's 10 ms. So by track 4, the track is delayed by 30 ms. The next step would be to make three short silent audio files. So I'd play "silent1" and then add track one to the channel 1 queue, play "silent2" and then add track 2 to the track 2 queue, play "silent3" and then add track 3 to the channel 3 queue, then play track 4.

The idea is to introduce a 30 ms delay to the first track, a 20 ms delay to the second, a 10 ms delay to the third, and then no delay to the fourth. By taking advantage of the idea that you can put audio clips in a queue on each channel, I should be able to find just the right delay to get the audio samples lined up.

The downside of this approach (aside from being a pain in the ass) is that there will be a 30 ms delay between pressing the "Play" button and the tracks all being heard.

If anyone has already lined up 4 tracks of audio to play at the exact same time via a different technique, please do share. If not, I'll post an example stack once I've got it figured out in case anyone else wants to make a music app.

[CALL-151]

Can't you accomplish the same thing by using "wait for x msec with messages"?

Still a clunky workaround (and both approaches my be fundamentally flawed due to variations in clock speed and background processes across deployment devices) but using "wait" seems simpler than creating those blank files.

[richardmac]

Can't you accomplish the same thing by using "wait for x msec with messages"?

Still a clunky workaround (and both approaches my be fundamentally flawed due to variations in clock speed and background processes across deployment devices) but using "wait" seems simpler than creating those blank files.

I tried to do that with a wait x ticks command, but the problem is that you can never get LiveCode to execute two iphone audio calls within a couple of ms of each other. Or at least I couldn't. If the tracks were, say, four different people speaking, it wouldn't have to be so synced. And if you are only doing two audio tracks, you might be able to fudge it. But with 4 tracks of music, there can't be a delay that people can hear.

So if the best you can get is, say, 10ms delay (I'm making that up,) then it doesn't matter if the channels all start playing at the same time - they can be a little off and you can compensate by starting them at ever so slightly different times.

You're right about the hardware differences. That sort of has me wondering if maybe it might be a better idea to somehow write a timecode loop myself, where I'm keeping track of the milliseconds and triggering events that way. The problem is that there's a lag when you send the iphone audio commands. Lots of thinking to do...

[richardmac]

OK, I tried this (pseudo code, where playCh1 is a routine to play a track on ch 1 via iphone audio call):

send playCh1 to this card in 2 seconds
send playCh2 to this card in 2 seconds
send playCh3 to this card in 2 seconds
send playCh4 to this card in 2 seconds

Didn't work. Stuff in the pendingMessages sits in an order, and that order is processed in order.

So I tried introducing a "wait 5 ticks" message in the routine for playCh1. All it did was make the pendingMessages queue wait an additional 5 ticks.

It's an interesting challenge.

[CALL-151]

Hmmm. I thought the point of wait...with messages was to allow subsequent messages to be processed during the wait. Sorry, I'm stuck for now. Hard to imagine that others haven't encountered this before, so hopefully someone has an elegant solution to offer.

[richardmac]

There's another aspect to this - the "next" option in iphonePlaySoundOnChannel is not fast enough for music application work. There is a small but noticeable pause when you use "next." When you use "looping," it's flawless. But this is a serious issue for anyone who wants to build anything musical. OK, I have another question but I'll start a new thread because it has nothing to do with this thread's issue.

[richardmac]

As a followup to this, I discovered a partial solution. When you issue an iPhonePlaySoundOnChannel command, it does two things - it creates the channel, and then it loads the sound and plays it. Do three or four of these in a row and you run into the issue where there's too much delay.

But if you use iPhoneSetSoundChannelVolume command first, and then later in the script use the iPhone play command, the lag between the commands is far less. In other words, this results in the three tracks playing very close together:

Code: Select all

 iphoneSetSoundChannelVolume "Ch1", "100"
 iphoneSetSoundChannelVolume "Ch2", "100"
 iphoneSetSoundChannelVolume "Ch3", "100"

 iPhonePlaySoundOnChannel daSound1, "Ch1","looping"
 iPhonePlaySoundOnChannel daSound2, "Ch2","looping"
 iPhonePlaySoundOnChannel daSound3, "Ch3","looping"

Whereas this results in a bigger delay between the three tracks:

Code: Select all

 iPhonePlaySoundOnChannel daSound1, "Ch1","looping"
 iPhonePlaySoundOnChannel daSound2, "Ch2","looping"
 iPhonePlaySoundOnChannel daSound3, "Ch3","looping"

Figured I'd share.

[CALL-151]

Can you now align the three tracks with something like this?

Code: Select all

iphoneSetSoundChannelVolume "Ch1", "100"
iphoneSetSoundChannelVolume "Ch2", "100"
iphoneSetSoundChannelVolume "Ch3", "100"

playSound3
playSound2
playSound1

command playSound3
   wait 20 milliseconds with messages
   iPhonePlaySoundOnChannel daSound3, "Ch3","looping"
end playSound3

command playSound2
   wait 10 milliseconds with messages
   iPhonePlaySoundOnChannel daSound2, "Ch2","looping"
end playSound2

command playSound1
   iPhonePlaySoundOnChannel daSound1, "Ch1","looping"
end playSound1

[bn]

Hi,

have you tried to prepare the chanel and the sound with the next parameter? From the iOS Release Notes it looks like it should speed up the start of the sound.

from iOS release notes:

iphonePlaySoundOnChannel sound, channel, type
Where sound is the sound file you wish to play, channel is the name of the channel to play it on and type is one of:
• now – play the sound immediately, replacing any current sound (and queued sound) on the channel.
• next – queue the sound to play immediately after the current sound, replacing any previously queued sound. If no sound is playing the sound is prepared to play now, but the channel is immediately paused – this case allows a sound to be prepared in advance of it being needed.

I did not test it but it sounds worth a try

Kind regards

Bernd

[richardmac]

Can you now align the three tracks with something like this?

Code: Select all

iphoneSetSoundChannelVolume "Ch1", "100"
iphoneSetSoundChannelVolume "Ch2", "100"
iphoneSetSoundChannelVolume "Ch3", "100"

playSound3
playSound2
playSound1

command playSound3
   wait 20 milliseconds with messages
   iPhonePlaySoundOnChannel daSound3, "Ch3","looping"
end playSound3

command playSound2
   wait 10 milliseconds with messages
   iPhonePlaySoundOnChannel daSound2, "Ch2","looping"
end playSound2

command playSound1
   iPhonePlaySoundOnChannel daSound1, "Ch1","looping"
end playSound1

You can, and I tried something similar, but it's no different then just issuing the commands one after the other. But like I said, it's OK - the delay is barely noticeable if you first create the channels by setting their volumes, like I pasted in the example. In my current project I've decided just use 3 tracks, and with that few tracks it's not bad. If you tried to do it with 8, 12, or 16 tracks it wouldn't work.

[richardmac]

Hi,

have you tried to prepare the chanel and the sound with the next parameter? From the iOS Release Notes it looks like it should speed up the start of the sound.

from iOS release notes:

iphonePlaySoundOnChannel sound, channel, type
Where sound is the sound file you wish to play, channel is the name of the channel to play it on and type is one of:
• now – play the sound immediately, replacing any current sound (and queued sound) on the channel.
• next – queue the sound to play immediately after the current sound, replacing any previously queued sound. If no sound is playing the sound is prepared to play now, but the channel is immediately paused – this case allows a sound to be prepared in advance of it being needed.

I did not test it but it sounds worth a try

Kind regards

Bernd

Yes, the next option works, but does not play the sample fast enough. In other words, say you have an audio clip of a drum beat that goes on for 8 beats. If you play it and use looping, it loops seamlessly. If you play it and use next, there's a very, very small pause when LiveCode plays the next clip - The pause is long enough that anyone can hear it and notice it. In music software, that's not acceptable - if you can hear the sample looping, it's no good. So you want to use looping for any serious audio work, and not the "next" option.

[richardmac]

Followup...

You can set the volume on three tracks and then play three tracks in a row, like I said earlier. But on close inspection (in this case the music app I'm building) by the third track the latency is really too much. So the trick to hacking around this issue is to time shift everything in the third track to be ever so slightly ahead of the beat. That way the latency is minimized. It also means, in a musical app, that you don't want to put any notes directly on the downbeat because you can't shift that back in time. It also means that the music can never be in "perfect" time because the device CPU will have an impact on how fast it can play tracks one after another.

To summarize, there are two major issues with trying to do any multichannel music for iOS in LiveCode - one is that there's no way to do any type of time code so you can't start 2 or more tracks at the EXACT same time. And two is that the "Next" option introduces too much latency to be musically useful (which I did submit as a bug.) There is a small workaround for the first problem and for the second you pretty much have to use "Looping," which works perfect.