Simultaneous movie and image presentation

Programming Help
1

Hi all,

I have a question regarding presentation of stimuli but I couldn’t find an answer online or in the Psychtoolbox Demos.

I am using Matlab R2021a and PTB 3.0.17 on a Windows 10 operating system.

What I want to achieve is to present a video that runs for a specific time and simultaneously flash additional static pictures but only for shorter durations. So for example I show a video for 30 seconds and additionally in the periods from 3-5sec., 8-11sec., 15-17sec. etc. I want to present images in addition to the actual movie.

You can also think about it as having two layers of display. On the lower layer (=background) the video runs while on the higher layer the images are shown in addition to the video.

I can show images and show a movie using Screen(DrawTexture) but I can’t show them at the same time. The code I have so far is the following:

 while ~KbCheck
        % Wait for next movie frame, retrieve texture handle to it
        tex = Screen('GetMovieImage', win, movie);
        
        % Valid texture returned? A negative value means end of movie reached:
        if tex<=0
            % We're done, break out of loop:
            break;
        end
        
        % Draw the new texture immediately to screen:
        Screen('DrawTexture', win, tex);
        
       % this would be a static image for example
        curr.image = imread('myimage.bmp');
        curr.texture = Screen('MakeTexture', win, curr.image);  
        Screen('DrawTexture',  win, curr.texture, [], []);
        
        
        curr.image = imread('pain.bmp');
        curr.texture = Screen('MakeTexture', win, curr.image);  
        Screen('DrawTexture',  win, curr.texture, [], []);        
        %Screen('Flip', win);
        %WaitSecs(5);
        
        % Update display:
        Screen('Flip', win);
        
        % Release texture:
        Screen('Close', tex);
    end

What this code does at the moment is show the video and from the beginning also show the image overlayed. But I can’t add more pictures or create a timeline for when those pictures need to be shown.

I would very much appreciate your help!

Thanks a lot

2

By default, if you are using Screen('GetMovieImage') then this will “block” waiting for the next frame, and your timing will be dictated by the frame rate of the movie. You could make your Flip times a multiple of the movie frame rate. But another solution requires not using blocking mode and using a second buffer texture. If Screen('GetMovieImage') returns a new frame, show that, if not, show the previous frame.

My movieStimulus class uses this technique (this is an OOP class and so me refers to itself):

me.blocking = false;
me.texture = Screen('GetMovieImage', me.sM.win, me.movie, me.blocking);
if me.texture > 0
	if ~isempty(me.buffertex) && ...
			me.buffertex > 0 && ...
			me.buffertex ~= me.texture && Screen(me.buffertex,'WindowKind') == -1
		try Screen('Close', me.buffertex); end
		me.buffertex=[]; 
	end
	Screen('DrawTexture', me.sM.win, me.texture, [], me.mvRect,...
		angle,[],[],[],me.shader);
	me.buffertex = me.texture; %copy new texture to buffer
elseif me.buffertex > 0
	Screen('DrawTexture', me.sM.win, me.buffertex, [], me.mvRect,...
		angle,[],[],[],me.shader)
end

Roughly it checks if texture is > 0 (a new frame): if so check if a previous buffertex is present and close it, then draw the new texture and assign it to buffertex | otherwise just draw buffertex. Now the timing is only dictated by Screen('Flip'), and so your other animations will work fine.You can see this in action easily with my toolbox:

m = metaStimulus;
m{1} = dotsStimulus('xPosition', 4, 'mask', true);
m{2} = movieStimulus('blocking', 0);
m.run; %test run of the stimuli

This creates a movie and an RDS stimulus, then runs them together at the FPS of the monitor display; the movie does not hold up the timing of the RDS…

3

Thank you for this very fast answer. I am not sure if I already get it but will try it out.

Another related question:

Is there any way to start a movie and let it run for a specific duration and while the movie is running present additional stimuli?

I did a similar thing using sounds which play in the background and programmed it that the sound should play until my n-th trial is finished. So it seems like the sound gets started and plays until my condition is reached. I would love to find a similar solution for my movies but I can’t.

Thanks a lot!