Clicking noise in sonification

Hi,
I’m trying to use mouse positions for audio output modulation by frequency (y-axis) and channel weighting (x-axis). There are several ways to produce sounds with PsychPortAudio but independent of using (FillBuffer, RefillBuffer, Master/Slave) there is always a clicking noise while moving the mouse i.e. changing sound frequency. My tests can be found here: GitHub - AndreasUniversityLuebeck/Sonification: Sonification of mouse movements. Is there a way to get a smooth sound transition initiated by mouse movements? Should I go towards OpenAL?

Testing environment: several computer, mostly MS Windows 10 (all updates), Matlab R2022b Update 3, Psychtoolbox 3.0.16 up to 3.0.18.12; audio hardware from built-in audio to Steinberg UR22; some rare tests on MacBooks with the same result as on Windows systems.

Greetings
Andreas

I think using PsychPortAudio(‘FillBuffer’,…) is the best way to go because it allows to overwrite the sample buffer on-the-fly even before all the samples are played. But, whatever you do, your updated signal must connect reasonably well to the currently playing signal, i.e. without major signal discontinuities at the connection point.
Here is some quick&dirty way of doing this, by connecting the signal chunks roughly at positive zero-crossings. With this code I can still hear faint brushing noise at higher frequencies and occasional clicks at low frequencies (even if I add 40ms more to the maxPrepSampCnt variable). Nevertheless, given the simple approach, it works pretty well.
I tested this on Win10 with some cheap onboard audio, so I wonder if you still hear occasional clicks when using a dedicated audio card.
Best,
Marc

[Code edited to fix the clicking problem - see my next post]

%--- Start from scratch regarding the audio module.
clear PsychPortAudio
%--- Initialize audio output. 
%    As far as I can see, the latency parameter of InitializePsychSound()  
%    is actually not used anymore anyway.
InitializePsychSound(0);    % 0: Standard, 1: Push hard for low latency
try
    %--- Open the audio interface.
    pahandle = PsychPortAudio('Open',[],[],2);  % reqlatencyclass=2 (= push for low latency, but not too hard)
    status = PsychPortAudio('GetStatus', pahandle);
    sampFreq = status.SampleRate;
    %--- Prepare the first tone, a dummy tone. The duration of this tone  
    %    defines the size of our audio buffer. 500ms should be long enough.
    bufferLenSecs = 0.5;
    toneVol = 0.5;
    toneFreqMin = 50;
    tone = MakeBeep(toneFreqMin, bufferLenSecs, sampFreq);
    %--- Create the audio buffer which, under the hood, will be used like a 
    %    ring buffer. The indexes into this buffer will be linear though, i.e.
    %    all the index modulo calculations are done by PsychPortAudio().
    PsychPortAudio('FillBuffer', pahandle, [tone; tone]*0);  
    %--- Start the playback in loop mode.
    %    Fiddling with 'RunMode' should not be necessary is not really necessary.
    %    According to the function's help text, the default mode is 1 anyway 
    %   (1 = keep things alive after playback has stopped). However, we will
    %   not stop playback anyway, unless the program has finished.
    PsychPortAudio('RunMode', pahandle, 1);     
    PsychPortAudio('Start', pahandle, 0);
    %--- 'maxPrepSampCnt' will account for the time, in terms of audio samples,  
    %    that will be needed for preparing the next audio buffer content. This includes   
    %    the overheads of the involved functions and should be an upper estimate.
    %    The low-level driver will read chunks of 'status.BufferSize' samples
    %    from the buffer, which would be the bare minimum for 'maxPrepSampCnt'.
    %    'status.BufferSize' is potentially (and should be) rather small, like 
    %    worth of a 10ms chunk or so - could be specified in the 'Open' call.
    maxPrepSampCnt = status.BufferSize + ceil(10e-3*sampFreq);
    %--- We cannot update the entire audio buffer, because we must not update samples
    %    that are needed for the ongoing playback. Therefore, we need to prepare
    %    an accordingly shorter tone. For each update, we will have to add to
    %    the update index, beyond 'maxPrepSampCnt', for updating the signal 
    %    at its next positive zero-crossing, which is what the 1/toneFreqMin term 
    %    accounts for.
    toneDuration = bufferLenSecs - maxPrepSampCnt/sampFreq - 1/toneFreqMin - 10e-3;
    %--- In order to easily compute the next best positive zero-crossing of the  
    %    currently played signal, we keep track of the buffer index where this
    %    signal has a positive zero-crossing.
    zeroCrossingIdx = 1;
    %--- Finally, the loop.
    screenRect = get(0, 'ScreenSize');
    screenWidth = screenRect(3);
    [mxOld, myOld, mb] = GetMouse();
    toneFreqOld = toneFreqMin;
    updateAlarm = GetSecs() + toneDuration - 200e-3;
    while ~any(mb)
        WaitSecs('YieldSecs',5e-3);
        [mx, my, mb] = GetMouse();
        if abs(mx-mxOld)>5 || abs(my-myOld)>5 || GetSecs()>updateAlarm
            %--- Update the tone according to the current mouse position.
            %    'tone' always starts with a positive zero-crossing.
            toneFreq = max(toneFreqMin,abs(my));
            tone = toneVol*MakeBeep(toneFreq, toneDuration, sampFreq);
            toneBal = mx/(screenWidth-1);
            %--- Get the current buffer playback sample index and compute  
            %    an (preliminary) update index that will leave us enough  
            %    time for completing the update.
            status = PsychPortAudio('GetStatus', pahandle);
            updateIdx = status.ElapsedOutSamples + maxPrepSampCnt;
            if updateIdx < zeroCrossingIdx
                continue
            end
            %--- Shift the update index forward to the next positive zero-crossing
            %    of the currently played signal so as to avoid audio artefacts.
            tonePeriodSampCntOld = sampFreq/toneFreqOld;
            tonePeriodCntOld = (updateIdx-zeroCrossingIdx)/tonePeriodSampCntOld;
            updateIdx = zeroCrossingIdx + round(ceil(tonePeriodCntOld)*tonePeriodSampCntOld);
            %--- Update the buffer. We use streamingrefill==2, which allows 
            %    overwriting samples which have not been played yet.
            %    Note that PsychPortAudio() uses 0-based indices, but this is
            %    a constant small offset we don't need to take care of here.
            PsychPortAudio('FillBuffer', pahandle, [tone*(1-toneBal); tone*toneBal], 2, updateIdx);
            %--- Prepeare the next update.
            updateAlarm = GetSecs() + toneDuration - 100e-3;
            mxOld = mx;
            myOld = my;
            toneFreqOld = toneFreq;
            zeroCrossingIdx = updateIdx;
        end
    end
    PsychPortAudio('Stop', pahandle, 0);
    PsychPortAudio('Close', pahandle);
    clear PsychPortAudio
catch e    
    clear PsychPortAudio
    rethrow(e);
end
1 Like

I fixed the clicking problem by basically adding these lines (also see the updated code in my previous post):

            if updateIdx<zeroCrossingIdx
                continue
            end

This defers updating the buffer in case the update comes so soon after the previous update that still some signal from even before the previous update has not played yet. In such a case I would base the new update on potentially wrong assumptions regarding the frequency of the currently played tone, because this frequency could be “older” than toneFreqOld.
Marc

Great, thank you! I will check and test :slightly_smiling_face: