Best practice: TTL event triggers in PTB stimulus presentation loop

Dear community,

I am working on an auditory experiment in which multiple short sounds (400ms long) are played in random order, which is controlled by a loop. For every sound onset, I wish to send an event marker (TTL pulse) to an EEG amplifier.

My code works nicely and the event markers arrive in the EEG, however I am wondering (1) if the code could be improved for jitter-free and minimally delayed timing, and, more specifically, (2) if good examples for the use of PTB’s IOPort() functions exist out there - I could not find any useful examples to work off from.

Here is my commented code:

% handle to the serial port
TB = IOPort('OpenSerialPort', 'COM3');

% set port to zero state
IOPort('Write', TB, uint8(0), 0);

% initialize the sound driver
InitializePsychPortAudio(1); % 1=push hard for low latency

% open the audio device
% default device, playback only, take full control of audio device, sampling frequency, stereo output
pahandle = PsychPortAudio('Open', [], 1, 2, 44100, 2);

% loop through random sequence and play back the respective sound
for i = 1:length(MyRandomSequence)

     % fill audio buffer (note this happens for every i in the loop - is there a better way?)
     PsychPortAudio('FillBuffer', pahandle, CellArrayContainingAllAudioData{MyRandomSequence(i)});

     % send TTL pulse to mark the event
     % 0 = no blocking, but my understanding is that then 'when' is not a meaningful time stamp?
     % if I set it to 1=blocking, the MATLAB freezes however and nothing happens
     [~, when] = IOPort('Write', TB, unit8(MyRandomSequence(i)), 0)

     PsychPortAudio('Start', 1, 0, 0);
end

PsychPortAudio('Stop', pahandle, 1);
PsychPortAudio('Close')

Ja, that approach is definitely wrong. You can search the forum, as I assume this has been discussed in the past. Or help PsychPaidSupportAndServices will tell you how to buy 30 minutes paid support by myself. If you fill out our user survey, which still has disappointing uptake, you’ll even get a discount code for some discount on it.

-mario

WELAQDUR-20221025164248:797e4e9ff65121ad6eb2ca3932f82ff940913c271380c3f4374263a2103b8e91

Thanks Mario. I would love to hear your thoughts on how to improve this. The requirements for the code are as follows: I have 8 individual sound files (400ms long). These need to be played in a specific order, which at the moment is given by a vector of numbers specifying the identity of the syllable (1 to 8) and the sequence in which they are to be played (e.g., 8 4 7 1 8 2 …). The onset of each syllable must be marked reliably with a TTL pulse. I don’t care much about portability of the script across devices and accommodation of different sound file types with different sampling rates or other parameters - this aspect of the task is highly controlled as we use a specific device with only the specified stimuli. What is, however, important, is precise timing.

Ok, your license key is now confirmed.

What you would do is the inverse order in your stimulus for-loop:

% Start one-time (1) playback asap (0), wait until predicted audio onset (1):
tOnset = PsychPortAudio('Start', pahandle, 1, 0, 1);
% Write TTL pulse, wait for confirmed transmit completion, so 'when' is meaningful:
[n, tTrigger(i)] = IOPort('Write', TB, uint8(MyRandomSequence(i)), 1);
% Returned n should be == 1 for successful trigger write.
% Estimate unwanted delay between sound onset and trigger onset 'latency':
latency = tTrigger(i) - tOnset;
% ...

% Wait for end of playback. A xruns > 0 suggests playback malfunction.
[~, ~, xruns] = PsychPortAudio('Stop', pahandle, 1);

This way the ‘Start’ command will compute the estimated time when sound leaves the audio output connector of your sound card, ie. the best estimate of true sound onset.
It will then pause your script until that time when sound truly starts. Immediately after that, IOPort sends the 1 marker-byte write command to the serial port and itself waits until the operating system confirms write completion and gives you a timestamp ‘tTrigger’ of that as well. If ‘n’ is not == 1 something went wrong during trigger write. ‘latency’ gives you an estimate of how much the TTL trigger was delayed wrt. sound onset.

The ‘Stop’ command waits until estimated stop/offset of sound, and reports ‘xruns’, which should be 0, or some audio playback glitch has been detected. After that you’re ready for the next trial loop iteration.

Making sound ‘Start’ wait for predicted onset is important, because there is always a software + hardware delay between when you ‘Start’ and when sound leaves the speakers, e.g., on MS-Windows with an onboard soundchip, this can be on the order of 10 - 20 msecs. Similar with the wait for IOPort, to account for system + hardware delays.

Now the quality of the audio time prediction depends on OS + audio driver + hardware, so you should always verify for you setup independently if possible, e.g., recording the sound wave and the TTL trigger with your EEG amplifier at least once to make sure that timing with your sound card is fine. A well working system, e.g., Linux or macOS, can achieve sub-milliseconds precision, ditto in the past for pro soundcards on MS-Windows. I don’t have current reference values at hand for MS-Windows.

There can be a write → emit delay for the TTL trigger as well, but i’d expect it to be sub-millisecond for typical real serial ports or no more than a millisecond for USB->Serial converters, at least as tested in the past with USB-serial converters from FTDI.

So all in all this method should get your TTL pulse to match within about 1-2 milliseconds on a well working system. It’s also advisable to use the Priority(MaxPriority(...)); to switch your script to realtime scheduling.

This is all good for sending isolated tones with triggers, because each Start->Trigger->Stop sequence will give good timing, but add a bunch of msecs delay between sounds, due to the hardware start delay before each sound. If you wanted to play a whole sequence of sounds back-to-back without any delay between them, you could define a sound schedule as demonstrated in BasicSoundScheduleDemo.m and only emit a trigger marking the start of the sequence, just as described above for a single sound.

If you needed a trigger per sound, and no gaps between sounds, you’d have to define a sound schedule, and then emit triggers at the time a sound is expected during scheduled playback, e.g., the code above for the first sound and trigger (i == 1), but then emit all other triggers at the expected time offsets since start of playback of the schedule, a la:

% Start one-time (1) playback asap (0), wait until predicted audio onset (1):
tOnset(1) = PsychPortAudio('Start', pahandle, 1, 0, 1);
% Write TTL pulse, wait for confirmed transmit completion, so 'when' is meaningful:
[n, tTrigger(1)] = IOPort('Write', TB, uint8(MyRandomSequence(1)), 1);

for i=2:length(MyRandomSequence)
    tOnset(i) = tOnset(i-1) + 0.400; % Next sound starts 400 msecs after previous one.
    WaitSecs('UntilTime', tOnset(i));
    [n, tTrigger(i)] = IOPort('Write', TB, uint8(MyRandomSequence(i)), 1);
end

Given that sound playback of a schedule, once started, is highly deterministic, e.g., an error accumulation of usually less than 50 microseconds audio clock drift per second of sound playback on typical sound hardware, probably more like < 15 usecs, this triggering at predicted times should be quite accurate, with minimal possible jitter of triggers wrt. sounds for a given operating system + driver + hardware combination.

Hope it helps.
-mario

[Time spent: Over 60 minutes, paid support membership of 30 minutes more than used up.]

1 Like

Hi all, I’m reviving this thread as we are once again working on a similar audio task with concurrent TTL triggering. We followed Mario’s advice above and use a sound schedule, which works well. In addition, we need to send a TTL trigger at every sound onset. The sounds are 100ms long pure tones created in Psychtoolbox using MakeBeep() with 20ms silence in between. We achieve this silence by adding the required number zero samples to the tones.

As was suggested above, we tried to send play the sound stream and send the first TTL pulse with the onset and then every next TTL deterministically when we expected a syllable would play. However, the drift between sound and TTL streams were too big so that at the end of the experiment TTLs were a little over an entire sound cycle too late.

We are now trying to start the sound schedule and send a TTL trigger when it starts and then another TTL trigger at the onset of each next tone. We define the sounds, then open the audio port. Next, this is the code we currently use for the playback and TTL:

% create the schedule and fill the buffer as much as possible
nfiles=128;
PsychPortAudio('UseSchedule', pahandle, 1);

for i = 1:nfiles
  PsychPortAudio('AddToSchedule', pahandle, buffer(i), 1, 0.0, 0.12, 1); % max buffer size is 128
end

ListenChar(-1);

% start the sound schedule and send a trigger to mark its onset
tOnset(1) = PsychPortAudio('Start', pahandle, 0, 0, 1); % get onset of the first sound in the playlist

[n, tTrigger(1)] = IOPort('Write', TB, uint8(trigger_list(1)), 2); %send the trigger marking the start of the sound stream

WaitSecs(0.001);

IOPort('Write', TB, uint8(0), 2); %reset the trigger box after a short wait

# determine nuisance latency between sound start and TTL marker
latency(1) = tTrigger(1)-tOnset(1);

# get info on sound stream
s = PsychPortAudio('GetStatus', pahandle);
sched_last = s.SchedulePosition;

% as long as sounds are playing, determine which sound is playing (position in stream) and send a corresponding TTL trigger
while s.Active == 1

  s = PsychPortAudio('GetStatus', pahandle);

  if s.SchedulePosition-sched_last == 1 %check whether next sound has started by looking at change in schedule position

  [n, tTrigger(s.SchedulePosition+1)] = IOPort('Write', TB, uint8(trigger_list(s.SchedulePosition+1)), 2);

  tOnset(s.SchedulePosition+1) = s.CurrentStreamTime; %define the onset time via sound status

  WaitSecs(0.001); % wait or else the port is sometimes reset before the trigger has been written

  IOPort('Write', TB, uint8(0), 2); %reset the port to zero state

  latency(s.SchedulePosition+1) = tTrigger(s.SchedulePosition+1) - tOnset(s.SchedulePosition+1);

  %add new sounds to the schedule until all sounds that must be played are in the schedule
  if
    s.SchedulePosition < ((12*sequenceLength+12 * (sequenceLength+ceil(sequenceLength/20))) - 128)

    PsychPortAudio('AddToSchedule', pahandle, buffer(128+s.SchedulePosition), 1, 0.0, 0.12, 1);
  end

  sched_last = s.SchedulePosition;
end

[keyIsDown,secs, keyCode] = KbCheck;

if keyCode(escapeKey) %abort experiment if escape key has been pressed
  break
end

WaitSecs('YieldSecs',0.001);
end

The issue here is that the difference between the onset of the sound and the sending of the trigger increases dramatically between the first and second sounds in the buffer, going from around 2-3 ms to about -15 ms, meaning that for the sounds in the while loop, the onset of the audio comes after sending the trigger, which makes little sense.

We think the issue lies in the two different approaches taken to define the onset of the sounds: the first sound’s timing is defined by PsychPortAudio(‘Start’), where we set the flags such that the code waits until the estimated time where sound has actually reached the speakers, whereas all other sounds have their onset defined via the sound status, which might not take this delay into account(?). Since the sound delay estimated by PTB on our setup is around 18 ms, which corresponds to the change in offset between the first and second sounds, this explanation is plausible to us.

We were wondering if this issue could be fixed by including a wait time of 18ms before sending each trigger in the while loop, or whether this would create other issues. Or is there maybe an approach using PsychPortAudio(‘GetStatus’) directly?

Any help would be appreciated, thanks for reading!

I guess you may not like to have extra hardware, but just in case you may consider, RTBox is ideal for EEG triggers. If you split the audio output to speaker and RTBox, the accuracy of the audio trigger won’t be affected by the audio driver or hardware delay. You will need to make a cable between RTBox to EEG port if the EEG system is not NeuroScan.

You would not want to just put 100 msecs audio + 20 secs silence into your buffers, but you’d add buffers with the 100 msecs audio into the sound schedule, followed by timing slots that instruct the schedule to pause playback of a following buffer until 120 msecs after the start of playback of the previous buffer. This gives “start playing 100 msecs buffer at time t0” then “pause playback (iow. silence) until t0 + 120 msecs” then “start playing next 100 msecs buffer at time t1 ~ t0 + 120 msecs” and so on…

That way the driver will schedule the start of each 100 msecs audio buffer individually with a spacing wrt. to the start of the previous buffer. The specified times all relate to host time aka Psychtoolbox GetSecs time, thereby this way time drift between host clock and sound card clock gets reset at start of each buffer, and error can’t accumulate. What will happen is some small jitter in the duration of the pauses between sounds, ie. it would be 20 msecs +/- some small error, typically less than 1 msec, on good systems in the small microsecond range. The approach is demonstrated in BasicAMAndMixScheduleDemo.m.

This way it should be possible to start a schedule, and then just have a loop in your script that just emits triggers at tOnset, tOnset + 120 msecs, tOnset + 240 msecs … and be reasonably accurate, as playback of individual buffers will also start at these times.

The limitation is that if you use such a timing slot to stop and the restart playback, there will be a minimum pause between stop and restart related to the lengths of a hardware audio buffer duration. So you need to configure the hardware for a low enough minimum latency if you need short pauses. On some operating systems (Windows) or sound hardware, pause durations of 20 msecs or less could be a problem at least with onboard sound chips.

Btw. the ‘UseSchedule’ subcommand allows to optionally specify how many slots the sound schedule should have. If 128 slots are not enough, you can specifiy a higher limit and avoid some of the complexity in your code.

Other than that, throwing hardware at the problem if Xiangrui Li’s latest RtBox can do this generation of triggers at onset of sounds, should be a robust solution if you only need to get triggers in sync with sounds, not make sure their onsets do not drift over time wrt. PTB GetSecs time, e.g., for visual stimulation etc.

For any further advice or clarification on this topic → PsychPaidSupportAndServices.