Unknown problem playing movies with Gstreamer on Ubuntu Linux

Programming Help
,
1

Hello,

I’m having problems getting movies to play with Psychtoolbox on my Ubuntu stimulus laptop. The same movie-playing script works fine on a different laptop running a mac operating system.

My script is supposed to play multiple 3-second movies in a row, without any gap between them.

On my Ubuntu laptop, the first tiny fraction of the first movie begins to play, but then it freezes for a few seconds, and the script crashes. When the script crashes, the PTB window closes, and it gives the error message:

PTB-ERROR: In Screen(‘GetMovieImage’) for movie 0: Timed out while waiting for new frame after 5 seconds!

Troubleshooting efforts so far

Running SimpleMovieDemo works fine.

Setting current directory to folder with movie files in it and running PlayMoviesWithoutGapDemo1 works, but there is a lag that occurs when the first frame of each movie starts. Basically, for each new movie that the script loops through, the first frame seems to load and then there is a pause of a couple of seconds before the rest of the movie plays. I’m not sure if this is expected behavior for this demo script or not.

Setting current directory to folder with movie files in it and running PlayMoviesWithoutGapDemo2 does not work. It plays the first movie in the directory just fine and then cuts to a black window and never plays the next movie. The script does not exit by itself so after killing the script and closing the screen I see that in the command window there were multiple PTB error messages saying the same thing:

PTB-ERROR: In Screen(‘GetMovieImage’) for movie 0: Timed out while waiting for new frame after 5 seconds!

Gstreamer is installed from the Ubuntu software manager.

I’m not sure what I should try next. I would really appreciate help troubleshooting this issue!

Thanks,
Sean

Version Information:

Operating system:
Ubuntu 22.04.1 LTS, codename jammy

Matlab version:
R2022b

Psychtoolbox Version:
3.0.18 - Flavor: Debian package - psychtoolbox-3 (3.0.18.12.dfsg1-1~nd22.04+1)

2

I solved the issue by forcing OpenMovie to load the movie files without sound, using the specialFlags1 option in Screen(‘OpenMovie’) set to 2.

I think everything is working properly now!

3

That’s a valid way to deal with it if you don’t need or have sound from your movies. Probably even a good idea if you know you don’t need movie sound playback, as it will require less processing resources inside GStreamer if all the audio infra is disabled.

Movie playback will block if sound output is blocked. Sound output can be blocked iff:

  • You use PsychPortAudio in parallel and it takes exclusive control of the same soundcard that is used by GStreamer to play movie sound. Can happen on Linux or MS-windows. Won’t happen on macOS, instead audio timing will be silently corrupted by the operating system and you just get wrong timing.

  • Whatever default soundcard used by GStreamer does not actually output sound. E.g., something like everything is set up to output via Headphones, but no headphones are plugged in. Or sound goes to the HDMI or Displayport sound output of your graphics card, but the connected HDMI/Displayport monitor does not actually have speakers etc.
    Seen on Linux and Windows, don’t know what macOS does in that case.

-mario

1 Like
4

Posting for the benefit of others.

I had the same issue:

PTB-ERROR: In Screen(‘GetMovieImage’) for movie 0: Timed out while waiting for new frame after 5 seconds!

when playing HDR (PQ-encoded) video (pixelFormat=11; % HDR-10). It happened when calling OpenMovie and CloseMovie several times.

There were also decoding errors in some videos - parts of the video had black (motion-compensated) blocks scattered across the frame.

Both issues were solved by setting:

specialFlags1 = 2 + 16; % do not play audio + convert all video textures immediately into the RGB format

Disabling audio alone did not solve the issue.

Psychtoolbox: '3.0.19 - Flavor: Manual Install, 11-Aug-2024 21:22:37'
GStreamer Core Library version 1.14.0
Nvidia 4090, 560 driver
5

So flag 2 fixed the timeouts/hangs, and flag 16 fixed the black blocks?
Needing 16 is weird, unless you apply some custom, non-standard texture drawing? 2 would be expected for setups without audio or such, as mentioned before.

On Linux or Windows? I assume the GStreamer version is 1.24.0, not 1.14.0?

6

Flag 2 did not fix either of the issues, only 16+2 did. I added “2” for a good measure.

Yes, we need to run a custom shader.

I was copying the video to a temporary texture before I discovered this flag 16. Copying alone worked fine: 2+16 + copy texture gave good video; 2 + copy texture caused freezing and blocky artifacts. Now I do 2+16 and draw directly to the screen (with a custom shader) and it also works fine.

On Ubuntu 22.04. Yes, sorry, this was a typo. It should be:

GStreamer 1.24.2
7

Ok, the need for flag 16 makes sense in that case. It is there for pretty much such purposes. If you only use Screen() functions for drawing textures, e.g., Screen('DrawTexture', ...)`` or builtin image processing functions, ie. ``Screen('TransformTexture', ...)`` to run shader based image processing, also utilizing single-pass or multi-pass sequences of builtin or custom made shaders and operators via CreateGLOperator()andAddToGLOperator()` then PTB will take care of conversion at the appropriate time for maximum efficiency. If you want to do more unusual stuff or your own shaders, then flag 16 allows to convert the movie texture immediately into a format friendly to 3rd party shaders.

E.g., for HDR footage an internal shader is used for PQ / HLG / whatever to linear decoding, and potential colorspace conversions are performed, to convert the planar HDR encoded texture into a standard RGBA floating point texture.

The need for flag 2 would be unrelated though, unless you need to fix some audio oddity, due to broken sound in the HDR movie, or some weirdness with your sound setup like trying to output to not connected speakers or headphones, or you use PsychPortAudio / Snd() / Beeper() or other sound output functions in parallel to movie playback, and want to avoid interference, or maybe some bug in sound handling - stuff that can cause hangs in playback unless properly coordinated.

GStreamer 1.24.2 on Ubuntu 24.04 I assume, as Ubuntu 22.04 does not ship with such a recent GStreamer version.

8

Ok, I need to follow up as not everything is working as expected.

Passing:

specialFlags1 = 2 + 16;

works perfectly fine on my development machine:

Ubuntu 22.04
'3.0.19 - Flavor: Manual Install, 11-Aug-2024 21:22:37'
gst-inspect-1.0 version 1.20.3
GStreamer 1.20.3

But when I run on a PC with a haploscope:

Ubuntu 24.04
'3.0.19 - Flavor: Manual Install, 11-Aug-2024 21:22:37'
gst-inspect-1.0 version 1.24.2
GStreamer 1.24.2

I am getting an error:

...
INTERNAL PSYCHTOOLBOX ERROR
	error:                Psycherror_OpenGL
	general description:  OpenGL call returned an error
	specific description: GL_INVALID_VALUE
	module name:          Screen
	subfunction call:     GetMovieImage
	file name:            /home/kleinerm/projects/OpenGLPsychtoolbox/Psychtoolbox-3/PsychSourceGL/Source/Common/Screen/PsychTextureSupport.c
	function name:        PsychCreateTexture
	line number:          158
...
Error using Screen
See error message printed above.

Error in hdrmfs_stereo_video/get_frame (line 66)
                tex(kk) = Screen('GetMovieImage', obj.disp.wins_unique(kk), obj.movie(kk), 1, [], 0);

complete log below.

At first, I thought it was a gstreamer issue - but the gst-play-1.0 executed from Matlab’s terminal (!gst-play-1.0 ...) worked fine on the same movie files. SimpleMovieDemo.m also worked fine.

When I pass instead:

specialFlags1 = 2;

the video plays on the PC with the haploscope, but it is missing the processing steps (as I use custom shaders).

When I Screen( 'DrawTexture' ...) to a temporary texture to trigger video processing, I am getting blocky artifacts, as reported before.

I could try Screen('TransformTexture', ...), but I cannot work out what transformProxyPtr is.

The complete log (note that some messages are from our software, please ignore those):

> stereo_video_demo
Loading HDRMFS geometric calibration profile from "/auto/homes/rkm38/projects/hdr_display/mfs_calibration/../hdrmfs_profiles/hdrmfs_camera_calib.mat"
Initializing display BL...
Initializing display BR...
Initializing display UL...
Initializing display UR...
Opening window for screen 2 ...


PTB-INFO: This is Psychtoolbox-3 for GNU/Linux X11, under Matlab 64-Bit (Version 3.0.19 - Build date: Jun 22 2024).
PTB-INFO: OS support status: Linux 6.8.0-48-generic Supported.
PTB-INFO: Type 'PsychtoolboxVersion' for more detailed version information.
PTB-INFO: Most parts of the Psychtoolbox distribution are licensed to you under terms of the MIT License, with
PTB-INFO: some restrictions. See file 'License.txt' in the Psychtoolbox root folder for the exact licensing conditions.

PTB-INFO: For information about paid support, support memberships and other commercial services, please type
PTB-INFO: 'PsychPaidSupportAndServices'.

PTB-INFO: Connected to NVidia TU116 [GeForce GTX 1650] GPU of NV-160 family with 4 display heads.

PTB-WARNING: Querying rasterbeam-position doesn't work on your setup! (Returns a constant value 0)
PTB-WARNING: This can happen if Psychtoolbox gets the mapping of connected displays to graphics card
PTB-WARNING: outputs wrong. See 'help DisplayOutputMappings' for tips on how to resolve this problem.



PTB-INFO: OpenGL-Renderer is NVIDIA Corporation :: NVIDIA GeForce RTX 4090/PCIe/SSE2 :: 4.6.0 NVIDIA 560.35.03
PTB-INFO: VBL startline = 2160 , VBL Endline = -1
PTB-INFO: Measured monitor refresh interval from VBLsync = 33.333450 ms [29.999895 Hz]. (29 valid samples taken, stddev=0.008395 ms.)
PTB-INFO: Reported monitor refresh interval from operating system = 33.333333 ms [30.000000 Hz].
PTB-INFO: Small deviations between reported values are normal and no reason to worry.
PTB-INFO: Support for OpenGL 3D graphics rendering enabled: depth-buffer and stencil buffer attached.

WARNING: Couldn't compute a reliable estimate of monitor refresh interval! Trouble with VBL syncing?!?


----- ! PTB - ERROR: SYNCHRONIZATION FAILURE ! -----

One or more internal checks (see Warnings above) indicate that synchronization
of Psychtoolbox to the vertical retrace (VBL) is not working on your setup.

This will seriously impair proper stimulus presentation and stimulus presentation timing!
Please read 'help SyncTrouble' for information about how to solve or work-around the problem.
You can force Psychtoolbox to continue, despite the severe problems, by adding the command
Screen('Preference', 'SkipSyncTests', 1); at the top of your script, if you really know what you are doing.




----- ! PTB - WARNING: SYNCHRONIZATION TROUBLE ! -----

One or more internal checks (see Warnings above) indicate that
queries of rasterbeam position are not properly working for your setup.

Psychtoolbox will work around this by using a different timing algorithm, 
but it will cause Screen('Flip') to report less accurate/robust timestamps
for stimulus timing.
Read 'help BeampositionQueries' for more info and troubleshooting tips.


PTB-INFO: Psychtoolbox imaging pipeline starting up for window with requested imagingmode 1057 ...
PTB-INFO: Will use 32 bits per color component floating point framebuffer for stimulus drawing. Alpha blending should work correctly.
PTB-INFO: Will use 32 bits per color component floating point framebuffer for stimulus post-processing (if any).
Opening window for screen 1 ...

PTB-WARNING: Querying rasterbeam-position doesn't work on your setup! (Returns a constant value 0)
PTB-WARNING: This can happen if Psychtoolbox gets the mapping of connected displays to graphics card
PTB-WARNING: outputs wrong. See 'help DisplayOutputMappings' for tips on how to resolve this problem.

PTB-WARNING: Couldn't even collect one single valid flip interval sample! Sanity range checks failed!
PTB-WARNING: Could be a system bug, or a temporary timing problem. Retrying the procedure might help if
PTB-WARNING: the latter is the culprit.

WARNING: VBL Calibration run No. 1 failed. Retrying...
PTB-WARNING: Couldn't even collect one single valid flip interval sample! Sanity range checks failed!
PTB-WARNING: Could be a system bug, or a temporary timing problem. Retrying the procedure might help if
PTB-WARNING: the latter is the culprit.

WARNING: VBL Calibration run No. 2 failed. Retrying...
WARNING: Will enable VBL busywait-workaround before trying final VBL Calibration run No. 3.
WARNING: This will hopefully work-around graphics driver bugs of the GPU sync-to-retrace mechanism. Cross your fingers!
PTB-WARNING: Couldn't even collect one single valid flip interval sample! Sanity range checks failed!
PTB-WARNING: Could be a system bug, or a temporary timing problem. Retrying the procedure might help if
PTB-WARNING: the latter is the culprit.


PTB-INFO: OpenGL-Renderer is NVIDIA Corporation :: NVIDIA GeForce RTX 4090/PCIe/SSE2 :: 4.6.0 NVIDIA 560.35.03
PTB-INFO: VBL startline = 2160 , VBL Endline = -1
PTB-INFO: Measured monitor refresh interval from VBLsync = 0.000000 ms [inf Hz]. (0 valid samples taken, stddev=10000000.000000 ms.)
PTB-INFO: Reported monitor refresh interval from operating system = 16.666667 ms [60.000000 Hz].
PTB-INFO: Small deviations between reported values are normal and no reason to worry.
PTB-INFO: Support for OpenGL 3D graphics rendering enabled: depth-buffer and stencil buffer attached.

WARNING: Couldn't compute a reliable estimate of monitor refresh interval! Trouble with VBL syncing?!?

WARNING: Mismatch between measured monitor refresh interval and interval reported by operating system.
This indicates massive problems with VBL sync.

PTB-WARNING: Unable to measure monitor refresh interval! Using a fake value of 16.666667 milliseconds.


----- ! PTB - ERROR: SYNCHRONIZATION FAILURE ! -----

One or more internal checks (see Warnings above) indicate that synchronization
of Psychtoolbox to the vertical retrace (VBL) is not working on your setup.

This will seriously impair proper stimulus presentation and stimulus presentation timing!
Please read 'help SyncTrouble' for information about how to solve or work-around the problem.
You can force Psychtoolbox to continue, despite the severe problems, by adding the command
Screen('Preference', 'SkipSyncTests', 1); at the top of your script, if you really know what you are doing.




----- ! PTB - WARNING: SYNCHRONIZATION TROUBLE ! -----

One or more internal checks (see Warnings above) indicate that
queries of rasterbeam position are not properly working for your setup.

Psychtoolbox will work around this by using a different timing algorithm, 
but it will cause Screen('Flip') to report less accurate/robust timestamps
for stimulus timing.
Read 'help BeampositionQueries' for more info and troubleshooting tips.


PTB-INFO: Psychtoolbox imaging pipeline starting up for window with requested imagingmode 1057 ...
PTB-INFO: Will use 32 bits per color component floating point framebuffer for stimulus drawing. Alpha blending should work correctly.
PTB-INFO: Will use 32 bits per color component floating point framebuffer for stimulus post-processing (if any).
Setting context for display BL ...
Setting context for display BR ...
Setting context for display UL ...
Setting context for display UR ...
Starting grid calibration
Starting eye-position calibration
Loading display geometry profile from: /auto/homes/rkm38/projects/hdr_display/mfs_calibration/../hdrmfs_profiles/hdrmfs_camera_calib.mat
Loading eye calibration profile from: /auto/homes/rkm38/projects/hdr_display/mfs_calibration/../eye_tracker/et_profile_test_observer
Finished grid calibration
Left eye position in RSB:  [163.784 150.383 847.343] mm
Right eye position in RSB: [228.065 154.28 848.833] mm
Inter-ocular distance:     64.2806 mm
=== Keyboard controls ===
UpArrow / DownArrow            - exposure
RightArrow / LeftArrow         - contrast
PageDown / PageUp              - change video
ESCAPE                         - exit
F1 (hold)                      - show shortcuts
/?                             - print shortcuts
INTERNAL PSYCHTOOLBOX ERROR
	error:                Psycherror_OpenGL
	general description:  OpenGL call returned an error
	specific description: GL_INVALID_VALUE
	module name:          Screen
	subfunction call:     GetMovieImage
	file name:            /home/kleinerm/projects/OpenGLPsychtoolbox/Psychtoolbox-3/PsychSourceGL/Source/Common/Screen/PsychTextureSupport.c
	function name:        PsychCreateTexture
	line number:          158


PTB-INFO: There are still 2 movies open. Screen('CloseAll') will auto-close them.
PTB-INFO: This may be fine for studies where you only use a single movie, but a large number of open
PTB-INFO: movies can be an indication that you forgot to dispose no longer needed movie objects
PTB-INFO: via a proper call to Screen('CloseMovie', moviePtr); , e.g., at the end of each trial. These
PTB-INFO: stale movies linger around and can consume significant memory and cpu ressources, causing
PTB-INFO: degraded performance, timing trouble and ultimately out of memory or out of ressource
PTB-INFO: conditions or even crashes of Matlab/Octave (in rare cases). Please check your code.



PTB-INFO: There are still 18 textures, offscreen windows or proxy windows open. Screen('CloseAll') will auto-close them.
PTB-INFO: This may be fine for studies where you only use a few textures or windows, but a large number of open
PTB-INFO: textures or offscreen windows can be an indication that you forgot to dispose no longer needed items
PTB-INFO: via a proper call to Screen('Close', [windowOrTextureIndex]); , e.g., at the end of each trial. These
PTB-INFO: stale objects linger around and can consume significant memory ressources, causing degraded performance,
PTB-INFO: timing trouble (if the system has to resort to disk paging) and ultimately out of memory conditions or
PTB-INFO: crashes. Please check your code. (Screen('Close') is a quick way to release all textures and offscreen windows)



INFO: PTB's Screen('Flip', 11) command seems to have missed the requested stimulus presentation deadline
INFO: a total of 4 times out of a total of 5 flips during this session.

INFO: This number is fairly accurate (and indicative of real timing problems in your own code or your system)
INFO: if you provided requested stimulus onset times with the 'when' argument of Screen('Flip', window [, when]);
INFO: If you called Screen('Flip', window); without the 'when' argument, this count is more of a ''mild'' indicator
INFO: of timing behaviour than a hard reliable measurement. Large numbers may indicate problems and should at least
INFO: deserve your closer attention. Cfe. 'help SyncTrouble', the FAQ section at www.psychtoolbox.org and the
INFO: examples in the PDF presentation in PsychDocumentation/Psychtoolbox3-Slides.pdf for more info and timing tips.



WARNING: This session of your experiment was run by you with the setting Screen('Preference', 'SkipSyncTests', 1).
WARNING: This means that some internal self-tests and calibrations were skipped. Your stimulus presentation timing
WARNING: may have been wrong. This is fine for development and debugging of your experiment, but for running the real
WARNING: study, please make sure to set Screen('Preference', 'SkipSyncTests', 0) for maximum accuracy and reliability.


WARNING: This session of your experiment was run by you with the setting Screen('Preference', 'SkipSyncTests', 1).
WARNING: This means that some internal self-tests and calibrations were skipped. Your stimulus presentation timing
WARNING: may have been wrong. This is fine for development and debugging of your experiment, but for running the real
WARNING: study, please make sure to set Screen('Preference', 'SkipSyncTests', 0) for maximum accuracy and reliability.
Error using Screen
See error message printed above.

Error in hdrmfs_stereo_video/get_frame (line 66)
                tex(kk) = Screen('GetMovieImage', obj.disp.wins_unique(kk), obj.movie(kk), 1, [], 0);
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in stereo_video_demo (line 77)
        [svid, vid_tex] = svid.get_frame();
9

Screen('Preference','ConserveVRAM', 512) aka kPsychAvoidCPUGPUSync in help ConserveVRAMSettings - it disables some of the OpenGL error checks. Maybe you get lucky and that OpenGL error does not have any consequences and stuff works fine enough with ignoring that OpenGL api error, who knows?

GStreamer or Ubuntu version is not that likely to be the reason.

Something is different between your development and haploscope machine. The NVidia driver version? The type and configuration of gpu’s? The X-Screen setup, or that you for some reason seem to open multiple onscreen windows, on different X-Screens / different gpu’s attached to those screens, instead of one X-Screen with two displays, as recommended for the dual-display haploscope? There are a lot of sync errors and other stuff that suggest a suboptimal multi-display setup → See my other responses in that other thread.

You do have a complex and unusual setup that seems to involve multiple NVidia gpu’s, and proprietary NVidia graphics drivers, something at least I don’t think I ever tested, at least not in the last 8 years.

The cause of the error could be some bug in you own code wrt. how you set up your custom shader, and one driver on one setup has more stringent error checking than the other, detecting or missing a real OpenGL misuse. Or it could be a graphics driver bug on your haploscope machine. In theory it could also be a PTB bug that only triggers on certain driver or gpu combos.

Without full reproducer code and extensive OpenGL api call tracing and correlation with the code that is impossible to say. You could sprinkle some gluErrorString calls into parts of your script to narrow down the location when the bug really happens. The message you’ve shown is just when an OpenGL error check is done, not the location of the bug.

I also don’t know why you get those blocky artifacts when drawing first into offscreen windows, or how you set up your offscreen windows, but again without full reproducer code that is impossible to say.

10

Without full reproducer code and extensive OpenGL api call tracing and correlation with the code that is impossible to say. You could sprinkle some gluErrorString calls into parts of your script to narrow down the location when the bug really happens. The message you’ve shown is just when an OpenGL error check is done, not the location of the bug.

That is fair. I created a simple example reproducing the issue when using two separate screens to play video (we need to use two screens because of two separate GPUs):

https://www.dropbox.com/scl/fi/9purfthy7xmw9a6xqxpxh/two_screen_movie_playback.zip?rlkey=z89gumr22laclcel48h9f9k4i&st=r676uss6&dl=0

When the flags are set:

    specialFlags1 = 2 + 16; % do not play audio + convert all video textures immediately into the RGB format

I get an error:

INTERNAL PSYCHTOOLBOX ERROR
	error:                Psycherror_OpenGL
	general description:  OpenGL call returned an error
	specific description: GL_INVALID_VALUE
	module name:          Screen
	subfunction call:     GetMovieImage
	file name:            /home/kleinerm/projects/OpenGLPsychtoolbox/Psychtoolbox-3/PsychSourceGL/Source/Common/Screen/PsychTextureSupport.c
	function name:        PsychCreateTexture
	line number:          158

When the flags are set:

   specialFlags1 = 2; % do not play audio

Everything works fine.

There is also no issue when only a single screen is used.

This looks like an issue with switching between different OpenGL contexts. I ran out of ideas about what I could do differently on my side.

I will try to reproduce blocky artifacts in a separate example.

11

What happens if you replace your ‘GetMovieImage’ calls in that reproducer by:

        Screen('GetWindowInfo', win_l);
        tex_l = Screen('GetMovieImage', win_l, movie_l);
        Screen('GetWindowInfo', win_r);
        tex_r = Screen('GetMovieImage', win_r, movie_r);

I think I found a bug in the routine that gets triggered by specialflag 16 in that it avoids explicitly switching to the OpenGL context of the to-be-converted texture. This as part of what might have been a not well enough thought through optimization. It would cause use of the wrong OpenGL context while performing part of the texture format and data conversion triggered by flag 16. But only in a multiple onscreen window scenario, and only with a specific sequence of drawing or flipping commands before the ‘GetMovieImage’ call, e.g., the sequence in your reproducer script. Unlikely I ever tested that specific scenario.

The error would also manifest differently, if at all, depending on operating system, windowing system, single vs. multiscreen, graphics driver type…

Those “pointless” Screen('GetWindowInfo', ...); calls would enforce switching to the correct OpenGL context, so as a workaround they might fix the specific problem.

12

Just to confirm that your work-around with Screen('GetWindowInfo', win_r); works.
Thank you for the help!

13

Good. Too bad the feedback didn’t come two days earlier, or a proper bug fix could have been folded into the new 3.0.19.16 release.

I’d still be interested in a reproducer for those “blocky artifacts” problem you’ve seen when drawing into offscreen windows first, to see if that is somehow expected, or if there’s some problem there.

Wrt. Screen('TransformTexture') btw., there are a bunch of demos that use it, e.g., FastFilteredNoiseDemo, ImageUndistortionDemo, ImageWarpingDemo, ImagingVideoCaptureDemo, ConvolutionKernelTest and FloatTexturePrecisionTest. The moglFDF() and moglmorpher() functions used by MorphDemo / MorphTextureDemo, FDFDemo use it.

The function is nice to do iterative multi-pass image processing via shaders, ie. 1 or 2 input images in → 1st shader transform → 2nd shader transform → … → processed image out. Taking care of implementing this multi-pass operation efficiently with minimal overhead and avoiding expensive/memory bandwith heavy gpu memory copies whenever possible, recycling textures if desired for saving memory or boosting performance further, etc. It can often save you the hassle of using lots of low-level glXXX operations for setting up such shader based processing. This uses the same machinery we have for implementing the various stimulus post-processing operations provided via the PsychImaging('AddTask', ...) for monoscopic or stereoscopic stimuli, just that it doesn’t define and apply operations globally to the whole stimulus image (triggered by Screen('Flip') or Screen('DrawingFinished'). Instead it allows you to apply such operations “locally” to specific textures or offscreen windows, spitting out new (or transformed) textures or offscreen windows.

Those gloperators are little containers defining the sequence of processing passes, ie. they contain usually a GLSL shader handles + possibly LUT textures + other parameters like required precision for each stage in the sequence etc. Helper functions like CreateGLOperator(), AddToGLOperator(), Add XXXXToGLOperator() for adding various XXXX functions allow to define the operators. help PsychGLImageProcessing for other useful bits. help ProceduralShadingAPI has info about specifics of single-pass shaders that can be directly used with Screen('DrawTexture').