Playing Transparent Videos via PTB

Note: thanks to @mariokleiner who found this solution.

Normal video file formats do not support transparency, however the advantages of transparency in a video would be that you could “composite” the object against a dynamic or changing background colour. As an example, imagine using animated cartoon fixation calibration marker for research in infants, where the background luminance may change, which can affect the calibration value. You could create seperate videos for every potential background, but this would be incredibly tedious. There are a few video formats that do support transparency (GIF [1bit] and WebM for example), however these do not play directly in PTB.

AVI movies encoded from PNG transparent image sequences do play without problem in PTB. To generate such files, we first need to generate a sequence of PNG files encoding each frame of the movie, then use GStreamer to convert these into a movie file.

As an example, here is a random animated transparent GIF (artist: Vince Mckelvie) we will try to play in PTB:

yy1

The first step is to extract the individual frames into PNG images; ImageMagick’s convert command [1] can do this for us easily:

convert -coalesce input.gif out_%02d.png

This creates 22 PNG files. The last frame is empty, so you should delete out_22.png.

Now we need to assimilate these PNG files into a movie file, and gstreamer has a command that can do that for us:

gst-launch-1.0 multifilesrc location=./out_%02d.png ! "image/png,framerate=24/1" ! pngparse ! avimux ! filesink location=out.avi

You can change the framerate by editing the 24/1 to a desired payback rate. The resultant out.avi can then be easily loaded into PTB using Screen('OpenMovie',windowPointer,'out.avi'), and as long as you use OpenGL blending [2], it plays with tranparency. This screenshot shows a single frame from PTB composited against a yellow background, rotated +45° and scaled to 5° in size:

frame

As the source was a GIF, only 1-bit transparency is used in this example, but PNG images actually supports 8-bit transparency so you could get much smoother edges if you were to use a PNG image source.


[1]: to install Imagemagick you can use apt in Linux: sudo apt-get install imagemagick; brew in macOS: brew install imagemagick; and there is even a package manager for Windows now: choco install imagemagick

[2]: The required blend modes are: Screen('BlendFunction', windowPointer, 'GL_SRC_ALPHA', 'GL_ONE_MINUS_SRC_ALPHA')

2 Likes