Delays in image presentation

Programming Help
,
1

Hi,

I am trying to program an experiment involving the serial presentation of png images taken from the same folder as the script shown below. Two images are presented per random series of 0, 4, 8 or 12 items representing red or blue versions of the same object. Subjects have to respond to each image by pressing left (blue images in DiagB) or right (red images in DiagR) until the series is completed. Subjects have a time window to respond to each item in the sequence after which they’re told that they’re too late.

I’ve got the exact same version of this script but with words instead of images. The word script works perfectly but the image script shows substantial delays in the presentation of each image in the sequence. The first few images appear on time but then the image onset is so late that the image and the “too late” feedback appear at the same time. Then the onset timings are completely screwed up for the rest of the sequence…

Below is a simplified version of the script to give you a sense of how images are shown. But I can also provide the full version if it’s not enough. You’ll also find the specs of my computer and OS. I need to stress that I’m not working in the full-screen mode yet as I’m currently developing the script. I got exactly the same Matlab warnings for the word-script and the image-script related to the window mode so no specific warning about the image script.

Lemme know if you need any further information!
Cheers
Nic

Here’s my script:

Screen(‘CloseAll’);
close all;
clear all;
sca

DLRF = 1; % if 1 = Blue L, Red R, if 2 = Blue R, Red L

%%Default settings for setting up psychtoolbox
PsychDefaultSetup(2);
screens=Screen(‘Screens’);
screenNumber=max(screens);
w = PsychImaging(‘OpenWindow’,screenNumber,,[0 0 1280 900 ]);

%%Download images

DiagR = {‘BEDr.png’,…
‘BELr.png’,…
‘BOEKr.png’,…
‘BOOTr.png’,…
‘BRIEVENBUSr.png’,…
‘BRILr.png’,…
‘BUSr.png’,…
‘CADEAUr.png’};

DiagB = {‘BEDb.png’,…
‘BELb.png’,…
‘BOEKb.png’,…
‘BOOTb.png’,…
‘BRIEVENBUSb.png’,…
‘BRILb.png’,…
‘BUSb.png’,…
‘CADEAUb.png’};

%%Randomize images and create red and blue lists
indices = randperm(length(DiagR));
DiagR = DiagR(indices);
DiagB = DiagB(indices);
DiagB1 = DiagB(1:4);
DiagB2 = DiagB(5:8);
DiagR1 = DiagR(1:4);
DiagR2 = DiagR(5:8);

%%generate image sequence length (0, 4, 8 or 12 items per sequence)
DiagTask = [0 0 4 4 8 8 12 12];
DiagRand = randperm(length(DiagTask));
DiagTask = DiagTask(DiagRand)';

%%generate image sequence length (0, 4, 8 or 12 items per sequence)
RespInst = [0 0 0 0 1 1 1 1];
RespRand = randperm(length(RespInst));
RespInst = RespInst(RespRand);

for i = 1:4
%Instruction screen
if RespInst(i) == 0
D1R(i) = DiagR1(i);
D1B(i) = DiagB1(i);
D2R(i) = DiagR2(i);
D2B(i) = DiagB2(i);
elseif RespInst(i) == 1
D1R(i) = DiagR1(i);
D1B(i) = DiagB1(i);
D2R(i) = DiagR2(i);
D2B(i) = DiagB2(i);
end
end

for i = 1:8

WaitSecs(1);

%Diagnostic task

if DiagTask(i) == 0
    DrawFormattedText(w,'...','center','center', 0);
    Screen('Flip',w);
    WaitSecs(2);
    cong(1)={'NoDT'};
    RTDiag(1)=NaN;
    CorDiag(1)=NaN;
    diagS={'NoDT'};
    colorS={'NoDT'};

elseif DiagTask(i) ~= 0

    if DiagTask(i) == 4
        diag = [];
        font = [];
        d = [0 0 1 1];
        f = [0 0 1 1];
        nn = length(d);

        keepset = [0 0 1 1;0 1 0 1];
        keepset = keepset(:,randperm(nn));
        diag = keepset(1,1:4)
        font = keepset(2,1:4)
    elseif DiagTask(i) == 8
        diag = [];
        font = [];
        for m = 1:2
            d = [0 0 1 1];
            f = [0 0 1 1];
            nn = length(d);

            keepset = [0 0 1 1;0 1 0 1];
            keepset = keepset(:,randperm(nn));
            diag = [diag keepset(1,1:4)];
            font = [font keepset(2,1:4)];
            clear keepset
        end
    elseif DiagTask(i) == 12
        diag = [];
        font = [];
        for m = 1:3
            d = [0 0 1 1];
            f = [0 0 1 1];
            nn = length(d);

            keepset = [0 0 1 1;0 1 0 1];
            keepset = keepset(:,randperm(nn));
            diag = [diag keepset(1,1:4)];
            font = [font keepset(2,1:4)];
            clear keepset
        end
    end
        for k = 1:DiagTask(i)
            if diag(k) == 0 && font(k) == 0 %then InducerInst1(i) red
                
                d1r=Screen('MakeTexture',w,imread(char(D1R(i)),'png'));
                rect = [0,0,200,200];
                newrectD = CenterRectOnPoint(rect,640,450);
                Screen('DrawTexture',w,d1r,[],newrectD);

                [VBL stimulusOnset]=Screen('Flip',w);
            
                [secs,keys,deltaSecs]=KbWait(-1,[],GetSecs+2);
                [keyIsDown KeyTime]=KbCheck();
               
                if keyIsDown == 0
                    DrawFormattedText(w,'TOO SLOW!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                elseif DLRF == 1 && find(keys) ~= 75 
                    DrawFormattedText(w,'WRONG!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                elseif DLRF == 2 && find(keys) ~= 68
                    DrawFormattedText(w,'WRONG!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                else
                    DrawFormattedText(w,'','center','center', 0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 1;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                end

                Screen('Close',d1r);

            elseif diag(k) == 0 && font(k) == 1 %then InducerInst1 blue 
                
                d1b=Screen('MakeTexture',w,imread(char(D1B(i)),'png'));
                rect = [0,0,200,200];
                newrectD = CenterRectOnPoint(rect,640,450);
                Screen('DrawTexture',w,d1b,[],newrectD);

                [VBL stimulusOnset]=Screen('Flip',w);
            
                [secs,keys,deltaSecs]=KbWait(-1,[],GetSecs+2);
                [keyIsDown KeyTime]=KbCheck();

                if keyIsDown == 0
                    DrawFormattedText(w,'TOO SLOW!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);       
               elseif DLRF == 1 && find(keys) ~= 68 
                    DrawFormattedText(w,'WRONG!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
               elseif DLRF == 2 && find(keys) ~= 75
                    DrawFormattedText(w,'WRONG!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
               else
                    DrawFormattedText(w,'','center','center', 0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 1;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                end

                Screen('Close',d1b);

            elseif diag(k) == 1 && font(k) == 0 %then DTS2 red

                d2r=Screen('MakeTexture',w,imread(char(D2R(i)),'png'));
                rect = [0,0,200,200];
                newrectD = CenterRectOnPoint(rect,640,450);
                Screen('DrawTexture',w,d2r,[],newrectD);

                [VBL stimulusOnset]=Screen('Flip',w);

                [secs,keys,deltaSecs]=KbWait(-1,[],GetSecs+2);
                [keyIsDown KeyTime]=KbCheck();

                if keyIsDown == 0
                    DrawFormattedText(w,'TOO SLOW!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                elseif DLRF == 1 && find(keys) ~= 75 
                    DrawFormattedText(w,'WRONG!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                elseif DLRF == 2 && find(keys) ~= 68
                    DrawFormattedText(w,'WRONG!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                else
                    DrawFormattedText(w,'','center','center', 0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 1;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                end

                Screen('Close',d2r);

            elseif diag(k) == 1 && font(k) == 1 %DTS2 blue

                d2b=Screen('MakeTexture',w,imread(char(D2B(i)),'png'));
                rect = [0,0,200,200];
                newrectD = CenterRectOnPoint(rect,640,450);
                Screen('DrawTexture',w,d2b,[],newrectD);

                [secs,keys,deltaSecs]=KbWait(-1,[],GetSecs+2);
                [keyIsDown KeyTime]=KbCheck();

                if keyIsDown == 0
                    DrawFormattedText(w,'TOO SLOW!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                elseif DLRF == 1 && find(keys) ~= 68 
                    DrawFormattedText(w,'WRONG!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                elseif DLRF == 2 && find(keys) ~= 75
                    DrawFormattedText(w,'WRONG!','center','center',0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 0;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                else
                    DrawFormattedText(w,'','center','center', 0);
                    RTDiag(k) = KeyTime - stimulusOnset;
                    CorDiag(k) = 1;
                    Screen('Flip',w);
                    WaitSecs(0.3);
                end

                Screen('Close',d2b);

            end
      end
end

end

And for the computer specs:

|Processor|Intel(R) Core™ i7-10850H CPU @ 2.70GHz 2.71 GHz|
|Installed RAM|16,0 GB (15,7 GB usable)|
|Device ID|F5454551-35A3-47B6-A981-055F22D65729|
|Product ID|00330-53495-46274-AAOEM|
|System type|64-bit operating system, x64-based processor|
|Pen and touch|No pen or touch input is available for this display|

Edition Windows 10 Pro
Version 22H2
Installed on ‎16/‎07/‎2021
OS build 19045.2728
Serial number PF2PHF0L
Experience Windows Feature Experience Pack 120.2212.4190.0
2

Surely you bought a paid PTB support membership after you got your last two answers for free, and after we told you that the best way to thank us is for your lab to buy a paid support membership, as that’s what keeps the lights on for Psychtoolbox (or not so much, as too few labs contribute). → help PsychPaidSupportAndServices for info how to buy.

So please post your paid authentication token for further help.
-mario

3

There you go: HE4AVH97-2023461232:4376c1c695f12eb3d044815155e1db0fb618d8098423391ea616564e9a04a2c8

Thanks for your help!

N.

4

Ok, finally your authentication token came through validation. You managed to buy your license just right before people here went on their easter vacation, hence a long delay…

There is one obvious reason for your script to behave like that in one condition. Much of the code is structured highly suboptimal, prone to facilitate making coding mistakes, e.g., by repeating lots of the drawing and flipping and response collection code over various if-else branches. This copy & paste & modify approach increases the likelihood or introducing some differences or mistakes into some if-else branches, but not others. So are you sure this delay/‘too late’ happens in all conditions and not just one condition, pointing to some bug in one of the branches?

Looking at it closer, the code after the elseif in this branch …

elseif diag(k) == 1 && font(k) == 1 %DTS2 blue

… is definitely wrong, compared to the others. It misses the Screen('Flip',...) statement after ‘DrawTexture’ and before KbWait! So whenever you’d hit that condition, you’d end with a “too late” situation. My guess would be that fixing that will fix it.

Other remarks:

Another problem for robustness and portability is your use of KbWait and then KbCheck, instead of simply a KbStrokeWait which would be more robust. Also you specify different keyboard device indices for KbWait vs. KbCheck. This may not matter on current MS-Windows with current Psychtoolbox, because Windows can’t differentiate between different keyboards, but it would likely break your script on other operating systems like Linux and macOS, or maybe with future PTB releases on Windows. Additionally, because your waiting for a keypress checks all connected keyboards, but the KbCheck only checks the default keyboard. E.g.,

[secs, keys] = KbStrokeWait(-1, stimulusOnset + 2);

would be much more robust. At least for fullscreen windows on a properly working system it would set the 2 seconds timeout relative to actual stimulus onset, consistently query all connected keyboards.

Then instead of if keyIsDown == 0 you can use if ~any(keys) for robustness.
You’d also rather use find(keys) ~= KbName('LeftArrow') or similar response keys, as those key codes which you hard-coded (68, 75) are MS-Windows specific and make your script non-portable to other operating systems.

For performance reasons, you’d also do the imread() and Maketexture calls before entering your trial loop, and the Close for the textures at the end of a session. and only have the DrawTexture and Flip inside the loop. At least with a limited set of target images which easily fits into system RAM, like in your case.

Assuming that fixing that obvious mistake in your code doesn’t solve it, more thoughts:

Is this a machine with Intel graphics? Windows + Intel is notorious for visual timing trouble.

Does the same problem happen in regular fullscreen mode? I wonder if something weird goes on between the desktop compositor and your scripts timing in windowed mode.

The way your script is written should make it impossible for a “too late” to happen, unless Screen(‘Flip’) returns long before the stimulus has actually shown on screen. Iow.

‘Flip’ returns, then KbWait waits for 2 seconds for a keypress, but the user does not see a target image, because ‘Flip’ returned too early, so user does nothing for at least 2 seconds and then gets a “too late”.

One of the problems of MS-Windows with DWM active, e.g., in windowed mode, and especially with broken Intel graphics even in fullscreen mode, is that timing/timestamping can be wrong and flip may return multiple video refresh cycles too early with too early timestamps. Errors in the +/- 50 msecs range on a 60 Hz display have been observed. But errors/too early exceeding 2 seconds is something new.

Hope the advice helps,
-mario

[Time spent: 55 minutes out of 30 minutes paid → Paid support for this membership license more than exhausted.]