Hi,
We are using Psychtoolbox on Linux in Matlab 2024. We want to play a sequence of videos (there may be over 1000 videos) without any noticeable break between the videos. Each video has 1920x1080 pixels frames and will be played at 30 frames per second on a 3840x2160 pixel resolution video monitor. Each video will be about 15 seconds. The way we draw a video is that we preload the video file in memory, make textures (Screen(‘MakeTexture’)) for each frame and then draw each frame (Screen(‘DrawTexture’)) sequentially at 30 fps. The problem we are facing is that between two videos, the loading and texture making takes 3-5 seconds, and this loading and texture making time is what we are trying to minimize to virtually zero. And in doing so, we also want to minimize any effect on timing of the played videos.
We tried to use Matlab’s parfeval to load and make textures in parallel with the drawing of textures, but for some reason, making of textures in parfeval does not work.
Has anyone tried to kind of double buffer texture making and texture drawing ? Short of preloading all videos in memory and premaking all the textures (prohibitively costly for memory usage), ss there anything else we could try to play movies without a noticeable break ?
Thanks in advance for any help
Hi saumil, have you checked out PlayMoviesWithoutGapDemo1 and PlayMoviesWithoutGapDemo2?
Keith
Thanks Keith, just looked at it, seems very relevant, we were not using movie functions, the async option was something we did not know. We will try this. Thanks again.
Keith, we have implemented the code from demo2 and it generally works very well. Thanks a lot for pointing us to those examples. In a trial, we are playing about 10 3 seconds movies (played at 30 fps) in a sequence. What we have noticed is that in 20% of the movies, a frame is dropped. For us, it is important to draw all the frames and if a frame is dropped to know which one was dropped, otherwise we will be out of sync with other physiology data. Is there a solution for this, meaning knowing when and which frame was dropped ? Thanks, Best, Saumil
Also, the computer is a current generation threadripper computer, so plenty fast. Video card is GTX 1060.
You can add the value +4 as the optional async flag parameter in Screen('Openmovie', win, moviename, async, ...); - ie. in PlayMoviesWithoutGapDemo2.m edit lines 64 and 142 to have the value 4 and 2+4 respectively. This will prevent dropping of frames if some part of the playback loop would be temporarily too slow. The downside being the gradual loss of audio-video sync if your movie also has an audio track, ie. visual falling behind audio.
Then you can use the usual Screen('Flip', ...) stimulus onset timestamps to detect delayed visual frame presentation and take that into account.
If otoh. audio-video sync is more important than dropping the occasional frame during movie playback, then you can look at the optional 2nd return value pts of Screen('GetMovieImage', ...). If the gap between successive movie presentation timestamps is bigger than expected for a movie of a given playback rate fps, ie. substantially larger than ~ 1/fps seconds, e.g., >> 0.0333 seconds in your 30 fps case, then that’s your cue for a movie frame that had to be dropped to keep audio and video playback in sync during a temporary slow down of your movie playback loop.
Thanks Mario, we made the suggested change and now everything is working well. We dont use audio at the moment, so are fine. Before making the above change, we also empirically verified that when difference between lastpts and pts (as in example code) exceeded the movie frame interval, it would indicate the time of the dropped frames. Most of the time, only one frame dropped randomly.
Good. Also remember the Priority() command for realtime optimizations. It is not used in many of our demos, but can still be quite a boost for deterministic timing in various scenarios, depending on hardware and task at hand.
Thanks Mario.
Also, is there a way to pause/resume a movie that is already playing ?
Screen('PlayMovie', movie, rate) allows to do that by setting different rates, PlayMoviesDemo.m demonstrates it.