We understand that Psychtoolbox works on Raspberry Pi hardware, and most features seem to work well on the Pi 4 (or 400). We are wondering if it is possible to achieve native high (>8) bit HDMI output to a compatible display with Psychtoolbox, Octave and Raspberry PI OS in their current versions.
We have had success with PsychImaging(‘AddTask’, ‘General’, ‘EnableNative10BitFramebuffer’) when we use hardware configurations that one might expect to work (e.g. Windows 11, Nvidia 2070, LG 32EP950 “true 10 bit” capable monitor). But on the Raspberry Pi 400 we get the usual error that we see on other systems (e.g. with Intel on-board graphics):
PTB-INFO: Trying to enable at least 10 bpc fixed point framebuffer.
PTB-ERROR: Your script requested a 30 bpp, 10 bpc framebuffer, but the OS rejected your request and only provides 8 bpc….
We understand that this is a little ambitious. If this isn’t something that works already, do you know if it is theoretically possible, and might it be implemented in the future?
We have had success using the Pi 400 with one of our Bits# systems, but we’re trying to achieve high bit-depth colour on a standard (but 10-bit capable) monitor without that.
nice to virtually meet you again. So I digged through the most recent driver code and this is the current state as of upcoming Mesa 23 and Linux 6.2:
Current RaspberryPi OS - or any other OS for RPi 4/400 do not support native RGB output with 10 bpc, only standard 8 bpc with any existing visual stimulation software.
Specifically there isn’t any display driver support code for creating of scanning out 10 bpc framebuffers, e.g., RGB8A2, regardless if OpenGL driven, Vulkan driven or any other rendering api.
What seems to already exist is support for scanout of 10 bpc YUV YCbCr planar buffers, as used for hardware accelerated video playback (specifically: 2 plane YCbCr420 content - 3 10 bit components and 2 padding bits packed into 4 bytes. 2x2 subsampled Cr:Cb plane 10 bits per channel packed). Know as FOURCC P030. There may be some 3rd party video players that could play back some 10 bpc content, e.g., ffmpeg, mpv or the kodi multi-media center software. But that doesn’t help with any visual stimulation software, only with playback of precreated HVEC movie content.
The gamma hardware LUT’s on the Pi4/400 still seem to be 8 bpc in → 8 bpc out only. I don’t know if they get properly bypassed in this YUV 10 bpc mode, or not, iow. if one really gets 10 bpc output. However, as far as I know, the SoC of the Pi is normally used as part of Smart TV’s, so it would not be too surprising if true 10 bpc are possible for consumer entertainment purposes.
There is a “bread crumb” of evidence - one hardware register programming constant in the code of the Linux 6.2 vc4 display driver - that the Broadcom VideoCore-6 gpu+display engines in the RPi 4/400 may be capable of RGB10A2 scanout, as would be usable for 10 bpc output by Psychtoolbox. But absolutely no driver code has been written to expose that functionality yet for display, or OpenGL rendering or Vulkan rendering.
Iff somebody would write such driver code, PTB would automatically pick it up and expose native 10 bit framebuffers over HDMI.
The OpenGL driver itself - and thereby current PTB on RaspberryPi OS - is capable of creating and rendering into 16 bpc floating point framebuffers, and even 32 bpc float framebuffers if one can do without alpha blending. That’s enough to render and post-process stimuli with > 10 bpc. They can be output as 8 bpc signal to VPixx visual stimulators or CRS Bits+ / Bits# etc. with the Mono++ / Color++ modes. Not at the speed of a regular desktop gpu, but good enough for some purposes i suppose. So one can pair a multi-thousand euro stimulator with a 50 Euro RPi 4 and make it work.
So this likely is a software limitation that could be fixed with enough work on the Linux kernel driver and Mesa OpenGL library, but the devil is always in the details. I was a contributor to the RPi1/2/3 video drivers to Linux myself in the past, mostly in the areas of general bug fixing and improvements, visual presentation timing and timestamping, and LUT’s – making the Pi’s fit for Psychtoolbox. I’d probably have the skills to try to explore such an improvement myself, but not the time, unless it is well paid for.
-mario
[80 minutes out of 30 paid minutes used up - All support exhausted multifold]