Here are the details of my particular setup:
PTB: 3.0.17 - Flavor: Manual Install, 19-Feb-2021 22:52:56
Matlab: R2020a Update 4 (9.8.0.1417392) maci64
OS: macOS 10.15.6
Hardware: MacBook Pro (16-inch, 2019); 2.6 GHz 6-Core Intel Core i7; 16 GB 2667 MHz DDR4; AMD Radeon Pro 5300M 4 GB + Intel UHD Graphics 630 1536 MB
Here’s the issue:
For certain .mov files, but not all, that I’m playing using PsychToolbox, the command Screen(‘CloseMovie’) causes Matlab to hang indefinitely. There are no errors or warnings printed to the Matlab console (and wrapping the call in a try/catch doesn’t change anything), and I don’t get any other indicators (spinning beachball of death), but 1) the next line of code (flipping a blank screen) never runs, and 2) Matlab stops interpreting keyboard input. For 2) in particular, the command line will still display what is being typed, but ctrl-c doesn’t have any effect, and script/function names aren’t interpreted—for instance, typing ‘sca’ and hitting return only has the effect of having ‘sca’ followed by a new line appear in the command window. When I try digging through Console at the time of the hang, nothing jumps out, but I also don’t know what to look for.
I’ve searched through as much of the documentation/help forum as I could find, and my guess is that the answer is basically here… that is, I’m pairing a buggy OS with an unsupported GPU. Unfortunately, this is for an fMRI study, and I really can’t change my setup at this point. My solution for now is to just not call Screen(‘CloseMovie’), which leaves all of the RAM used by the end of the experiment (I play 9 clips totalling about 30 minutes), but at least avoids crashes.
So this is mostly just a bug report, not request for help, but if anyone has guidance on messages I can watch for in Console to give more insight into what’s going on, it would be appreciated. Thanks.
Is there something specific about how the movies that work and fail are different? Length, codec, etc? Is it always the same movies?
1 Like
The clips are all about the same length (between 3–4 minutes), and all encoded using H.264. The clips do differ in terms of their aspect ratios and frame rates. The success/failure of Screen(‘CloseMovie’) is consistent in terms of which clips hang or don’t.
I can try to do some testing as to whether there is anything systematic in terms of properties of the clips and hanging. If it would be helpful from a debugging standpoint to collect other information (for instance, the value of the final texture), I can try doing that as well.
@dcnieho advice is correct. This could be a peculiar GStreamer codec bug, but more likely some of your movies are misencoded, ie. they do something not really supported by the H.264 standard, or at least not the parts of the standard supported by GStreamers codec. There are many ways to get this wrong, and different codecs are more or less lenient towards encoding violations, e.g., using unsupported aspect ratios or combos of width x height.
Extra fun is to be had if your gpu claims to support h264 hw decoding and gstreamer hands off some of the work to the gpu driver, but the gpu then chokes on specific peculiarities of a non-standard movie. That’s when things start/stop working depending on OS / OS version / graphics card.
My experience is that Apple macOS Quicktime is especially bad at dealing with many formats that other players don’t have problems with, so if macOS would get involved in the task, that could create extra fragility.
Try encoding all of them in exactly the same format, and picture size as the ones that work fine.
-mario
Holy crap that’s a nightmare. I’ve sized the videos very specifically to control for various visual confounds. They’re all approximately typical aspect ratios… but not exactly, because of the need to have multiples of two for width/height. So that issue being the culprit seems very plausible. I could potentially try re-encoding the videos with letter/pillar/windowboxes the same color as my default background, to resize all videos to a uniform supported aspect ratio.
The other thought I had was, sorry if this is trivial, but is there a way to just treat the movies differently? Like, extract/preload the textures, draw/flip them at the correct frame rate, and at the same time play the audio? Not sure if that’s exactly possible or if it would circumvent this issue, but it was the other obvious solution that came to mind that wouldn’t require resizing.
Thanks both for your input (and @mariokleiner for all your dev/maintenance work!).
Ben, please consider supporting Mario (https://www.psychtoolbox.net/) by buying priority support.
Be careful with a letterbox around the video, as depending with pixelformat settings these may come out just wrong when encoded and thus be visible against your background.
Given what Mario says, it may be worth trying to force hardware decoding on or off. I bet there is some GStreamer magic incantation to force either, if possible, but i do not know of it. Trying a different encoder (ffmpeg for instance provides multiple for H.264) may be worth experimenting with either. You’re “lucky” its always the same videos, as that at least makes debugging tractable.
There is a great ffmpeg GUI called FF-Works for macOS that makes re-encoding with a consistent codec for lots of videos easier: https://www.ffworks.net – or learn to use ffmpeg directly in a script…
You can use something like media-info (brew install media-info) to get the detailed info on each video…