Using toolbox with Big Sur and M1 MacBook?

:+1:

In my opinion, that’s how everybody should use Apple computers for neuroscience only, for entertainment and teaching use only - and even that only if one finds masochism entertaining and educational. Certainly not for serious data collection.

Ofc. that’s a bit of cheating, but the good kind of cheating, as you are not actually running Psychtoolbox on macOS with Apple M1, but you are running Psychtoolbox for Linux in a VM. Although ofc. again a Linux in a VM is only useful for teaching purpose, not for data collection. Does graphics acceleration work in this scenario? Or just software rendering?

Can your provide more info about your setup, maybe for the benefit of others? What Linux VM do you use? The NeuroDebian VM or something else custom made? I assume only a Linux distribution for 64-Bit ARM (aarch64) will work, as macOS 11 Big Sur’s Rosetta2 code translator doesn’t work for software in a VM, as far as i understood? Hence also why Matlab for Intel processors doesn’t work, because it would be untranslatable Intel code in a VM.

I am surprised a bit that PTB for Matlab or Octave on macOS would not work at all. Some threads on the Mathworks forum suggest that R2020b Update 4 or later for Intel Macs can work under Rosetta2 emulation? I’d assumed that would translate into running PTB for IntelMacs as well. My concern was more that macOS 11 is even more buggy than 10.15 from what i’ve read, and the security measures that already cause lots of trouble on 10.14/10.15 have been made even worse. Also no PsychtoolboxKernelDriver for Apples proprietary gpu, so essentially useless for actual data collection.

-mario

Apologies for the delayed reply here, Mario! Yes, I concede this is definitely cheating - and no, I will never be running actual experiments on my Mac, it’s just good to be able to practice at home (especially as a PhD candidate who is new to the field).

Does graphics acceleration work in this scenario? Or just software rendering?

At this stage, I believe that the graphics acceleration works OK, but I’ve not tried anything particularly complicated yet.

Can your provide more info about your setup, maybe for the benefit of others? What Linux VM do you use? The NeuroDebian VM or something else custom made? I assume only a Linux distribution for 64-Bit ARM (aarch64) will work, as macOS 11 Big Sur’s Rosetta2 code translator doesn’t work for software in a VM, as far as i understood?

I’m running a Linux VM using Parallels, which is a bit of a bandaid fix for now because when I set it up, it was the only VM I could find that worked fine on M1.

Hence also why Matlab for Intel processors doesn’t work, because it would be untranslatable Intel code in a VM.

Yes, that’s why I’m running Octave in the VM. Matlab works fine on the Mac, but the Linux version obviously only works via a regular Intel chip.

I am surprised a bit that PTB for Matlab or Octave on macOS would not work at all. Some threads on the Mathworks forum suggest that R2020b Update 4 or later for Intel Macs can work under Rosetta2 emulation?

As I said above, Matlab itself works completely fine on the M1, as far as I can tell. It’s definitely possible that psychtoolbox will too, but in my initial attempts to get it set up I ran into a variety of different issues, and having consulted all your help documentation and this forum, it seemed like the quickest solution might be just to get going with a VM - I was totally happy with a bandaid fix at the time.

I am hoping to try again with getting PTB working on the Mac itself (no VM) eventually, but as the current solution suits my current purposes, I don’t see the need to too much.

Apologies for any issues with my use of technical terminology in this reply - I’m very new to both psychophysics and VMs, so I’ve done my best to answer properly but it’s possible I’ve not explained exactly as you hoped.

Please let me know if you have any follow up questions.

I agree that this is a good use of a Linux VM on macOS. For training/education instead of data collection, this might be even better than using PTB on macOS directly, given all the security headaches imposed by Apple. And you already learn to use a real operating system for actual data collection.

One could even provide and share a suitable VM image for plug & play use.

Does graphics acceleration work in this scenario? Or just software rendering?

At this stage, I believe that the graphics acceleration works OK, but I’ve not tried anything particularly complicated yet.

Ah their product page suggests OpenGL 3 support on Linux. That’d be good enough for all the basics. On macOS we can only use OpenGL 2.1 due to Apples primitive OpenGL implementation.

Can your provide more info about your setup, maybe for the benefit of others? What Linux VM do you use? The NeuroDebian VM or something else custom made? I assume only a Linux distribution for 64-Bit ARM (aarch64) will work, as macOS 11 Big Sur’s Rosetta2 code translator doesn’t work for software in a VM, as far as i understood?

I’m running a Linux VM using Parallels, which is a bit of a bandaid fix for now because when I set it up, it was the only VM I could find that worked fine on M1.

Oh i just meant, which Linux distribution did you use? Ubuntu for RaspberryPi / ARM? Something else?

I am surprised a bit that PTB for Matlab or Octave on macOS would not work at all. Some threads on the Mathworks forum suggest that R2020b Update 4 or later for Intel Macs can work under Rosetta2 emulation?

No, i think this should be helpful to others. Thanks for taking the time and reporting back.

-mario

Oh i just meant, which Linux distribution did you use? Ubuntu for RaspberryPi / ARM? Something else?

Apologies - yes, I am on Linux Ubuntu via Parallels. :slight_smile:

1 Like

Just in case it’s at all useful. I just tried to run GarboriumDemo on my M1 MacBook Air in MATLAB R2019b with no joy:

Will draw 200 gabor patches per frame.
PTB-INFO: Retina display. Enabling panel fitter for scaled Retina compatibility mode.

PTB-INFO: This is Psychtoolbox-3 for Apple OS X, under Matlab 64-Bit (Version 3.0.17 - Build date: Dec 17 2020).
PTB-INFO: OS support status: OSX version 11.0 is not yet tested or officially supported at all for this Psychtoolbox release.
PTB-INFO: Type ‘PsychtoolboxVersion’ for more detailed version information.
PTB-INFO: Most parts of the Psychtoolbox distribution are licensed to you under terms of the MIT License, with
PTB-INFO: some restrictions. See file ‘License.txt’ in the Psychtoolbox root folder for the exact licensing conditions.

PTB-INFO: For information about paid priority support, community membership and commercial services, please type
PTB-INFO: ‘PsychPaidSupportAndServices’.

PTB-WARNING:PsychOSFixupFramebufferFormatForTiming: On screenId 0 no suitable mode with targetBpc 8. No-Op. Timing will suck!

PTB-ERROR[CGLSetFullScreenOnDisplay failed: invalid fullscreen drawable]. Main Screen() context

PTB-ERROR[Low-Level setup of window failed]:The specified gpu + display + gfx-driver combo may not support some requested feature.

Cool. That failing function is critical to get any visual stimulus onset timing or timestamping precision. Without it, it is game over for precise timing or good performance.

What happens if you execute Screen('Preference','ConserveVRAM', 16384); first? That would use the fallback path with broken timing and low performance/high latency, but may at least get a window up for entertainment and educational purposes.

-mario

Also, ``Screen(‘Preference’,‘Verbosity’,5); output of a failing run, ie. without the ‘ConserveVRAM’ setting from my last post, would be interesting for some more damage assessment.

-mario

Just got a M1 MacBook which, like others here, would be used for at-home debugging and code development. Under normal circumstances I receive the same error message as others here, but I can confirm that the Screen('Preference','ConserveVRAM',16384) command does allow stimuli to show on-screen. Interestingly though, KbCheck and KbWait appear to be broken in Big Sur / M1 – they don’t detect actual keypresses, but occasionally they return a keyCode which translates KbName(keyCode) as ‘LeftGUI’. I’m not able to work out exactly what causes LeftGUI to be returned, but it doesn’t seem to be anything to do with keyboard input.

Here’s my output with level 5 verbosity (without the ConserveVRAM override), in case it’s helpful:

>> Screen(‘Preference’,‘Verbosity’,5);
>> PsychImaging(‘OpenWindow’,max(Screen(‘Screens’)))
PTB-DEBUG: PsychGetScreenDepths(): mode 0 : w x h = 800 x 600, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 1 : w x h = 825 x 525, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 2 : w x h = 840 x 525, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 3 : w x h = 1024 x 640, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 4 : w x h = 1024 x 768, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 5 : w x h = 1152 x 720, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 6 : w x h = 1280 x 800, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 7 : w x h = 1280 x 800, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 8 : w x h = 1440 x 900, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 9 : w x h = 1440 x 900, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 10 : w x h = 1650 x 1050, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 11 : w x h = 1680 x 1050, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 12 : w x h = 2048 x 1280, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 13 : w x h = 2560 x 1600, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 14 : w x h = 640 x 480, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 15 : w x h = 720 x 450, fps = 60.000000, depth = 24
PTB-INFO: Retina display. Enabling panel fitter for scaled Retina compatibility mode.

PTB-INFO: This is Psychtoolbox-3 for Apple OS X, under Matlab 64-Bit (Version 3.0.17 - Build date: Apr 18 2021).
PTB-INFO: OS support status: OSX version 11.0 is not yet tested or officially supported at all for this Psychtoolbox release.
PTB-INFO: Type ‘PsychtoolboxVersion’ for more detailed version information.
PTB-INFO: Most parts of the Psychtoolbox distribution are licensed to you under terms of the MIT License, with
PTB-INFO: some restrictions. See file ‘License.txt’ in the Psychtoolbox root folder for the exact licensing conditions.

PTB-INFO: For information about paid priority support, community membership and commercial services, please type
PTB-INFO: ‘PsychPaidSupportAndServices’.

PTB-DEBUG: Display screen [0]:
PTB-DEBUG: (now: 1440x900@2@d=8@60Hz)
PTB-DEBUG: Allowed modes:
2560x1600@d=8@60Hz 1280x800@2@d=8@60Hz
2048x1280@d=8@60Hz 1024x640@2@d=8@60Hz
1650x1050@d=8@60Hz 825x525@2@d=8@60Hz
1680x1050@2@d=8@60Hz
1440x900@2@d=8@60Hz
720x450@2@d=8@60Hz 1440x900@d=8@60Hz
1280x800@d=8@60Hz
1152x720@d=8@60Hz
1024x768@d=8@60Hz
840x525@d=8@60Hz
800x600@d=8@60Hz
640x480@d=8@60Hz

PTB-DEBUG:PsychOSFixupFramebufferFormatForTiming: screenId 0 in unsuitable format 8 != 4 for 8 bpc.
PTB-DEBUG:PsychOSFixupFramebufferFormatForTiming: Trying to switch to suitable mode with format 4.
PTB-WARNING:PsychOSFixupFramebufferFormatForTiming: On screenId 0 no suitable mode with targetBpc 8. No-Op. Timing will suck!
PTB-INFO: Current backbuffer scaling src size 0 x 0 versus estimated display native / actual backbuffer size 2880 x 1800.
PTB-INFO: Screen 0 - Display unit 0x0, vendorId 0x610, modelId 0xa03c, width 286 mm.
PTB-INFO: Screen 0 - Apple MacBookAir10,1 builtin display of width 286 mm detected. Searching for database match…

PTB-ERROR[CGLSetFullScreenOnDisplay failed: invalid fullscreen drawable]. Main Screen() context

PTB-ERROR[Low-Level setup of window failed]:The specified gpu + display + gfx-driver combo may not support some requested feature.

PTB-DEBUG: Display screen [0]:
PTB-DEBUG: (now: 1440x900@2@d=8@60Hz)
PTB-DEBUG: Allowed modes:
2560x1600@d=8@60Hz 1280x800@2@d=8@60Hz
2048x1280@d=8@60Hz 1024x640@2@d=8@60Hz
1650x1050@d=8@60Hz 825x525@2@d=8@60Hz
1680x1050@2@d=8@60Hz
1440x900@2@d=8@60Hz
720x450@2@d=8@60Hz 1440x900@d=8@60Hz
1280x800@d=8@60Hz
1152x720@d=8@60Hz
1024x768@d=8@60Hz
840x525@d=8@60Hz
800x600@d=8@60Hz
640x480@d=8@60Hz

PTB-DEBUG:PsychOSFixupFramebufferFormatForTiming: Called on screenId 0 with fix disable request while fix is already disabled. No-Op.
Error using Screen
See error message printed above.

Error in PsychImaging (line 2330)
[win, winRect] = Screen(‘OpenWindow’, screenid, clearcolor, winRect, pixelSize, numbuffers, stereomode, multiSample,
imagingMode, specialFlags, clientRect, fbOverrideRect, vrrParams);

Ok, so it can be used in toy-mode at least. What’s the output of a successfull run with the ConserveVRAM setting and also Screen verbosity 10?

Could be the wrong HID device is queried as “default keyboard”. Apple broke this multiple times in the last years, the last time when they introduced the useless and annoying touchbar gimmick. Maybe we need another fix for M1 brokeness.

Does KbStrokeWait(-1) work? Output of [a,b,c] = GetKeyboardIndices and of PsychHID('Devices', -1) ?

Thanks. That output looks really bad, but lets see how the output looks with the workaround applied.

-mario

Thanks Mario. Given that the ConserveVRAM workaround allows a screen to launch, the keyboard detection issue is the only thing currently preventing use of PTB in ‘toy mode’ for basic debugging on a Mac.

KbStrokeWait(-1) doesn’t work – I just see letters output in the console, but the input isn’t recognised by PTB.

[a,b,c] = GetKeyboardIndices returns:

a: 9
b: ‘Apple Internal Keyboard / Trackpad’
c: struct containing entries for (let me know if you need any specifics): usagePageValue, usageValue, usageName, index, transport, vendorID, productID, version, manufacturer, product, serialNumber, locationID, interfaceID, totalElements, features, inputs, outputs, collections, axes, buttons, hats, sliders, dials, wheels, touchDeviceType, maxTouchpoints.

PsychHID('Devices', -1) returns 9.

Output below. Note that it’s also necessary to use SkipSyncTests, which is fine as far as I’m concerned when doing basic code debugging at home.

\>\> Screen('Preference','Verbosity',10);
\>\> Screen('Preference','ConserveVRAM',16384);
\>\> Screen('Preference','SkipSyncTests',1);
\>\> PsychImaging('OpenWindow',max(Screen('Screens')))
PTB-DEBUG: PsychGetScreenDepths(): mode 0 : w x h = 800 x 600, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 1 : w x h = 825 x 525, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 2 : w x h = 840 x 525, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 3 : w x h = 1024 x 640, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 4 : w x h = 1024 x 768, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 5 : w x h = 1152 x 720, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 6 : w x h = 1280 x 800, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 7 : w x h = 1280 x 800, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 8 : w x h = 1440 x 900, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 9 : w x h = 1440 x 900, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 10 : w x h = 1650 x 1050, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 11 : w x h = 1680 x 1050, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 12 : w x h = 2048 x 1280, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 13 : w x h = 2560 x 1600, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 14 : w x h = 640 x 480, fps = 60.000000, depth = 24
PTB-DEBUG: PsychGetScreenDepths(): mode 15 : w x h = 720 x 450, fps = 60.000000, depth = 24
PTB-INFO: Retina display. Enabling panel fitter for scaled Retina compatibility mode.


PTB-INFO: This is Psychtoolbox-3 for Apple OS X, under Matlab 64-Bit (Version 3.0.17 - Build date: Apr 18 2021).
PTB-INFO: OS support status: OSX version 11.0 is not yet tested or officially supported at all for this Psychtoolbox release.
PTB-INFO: Type 'PsychtoolboxVersion' for more detailed version information.
PTB-INFO: Most parts of the Psychtoolbox distribution are licensed to you under terms of the MIT License, with
PTB-INFO: some restrictions. See file 'License.txt' in the Psychtoolbox root folder for the exact licensing conditions.

PTB-INFO: For information about paid priority support, community membership and commercial services, please type
PTB-INFO: 'PsychPaidSupportAndServices'.

PTB-INFO: Using Cocoa + No display capture / compositor lockout for fullscreen window. Timing will be disastrous.
PTB-INFO: Using Cocoa for fullscreen windows to work around graphics driver bugs or limitations of OSX.
PTB-INFO: Presentation timing precision is likely disastrous on most machines. Check your results.
PTB-DEBUG: PsychCocoaCreateWindow(): On other thread.
PTB-INFO: Using GLEW version 2.1.0 for automatic detection of OpenGL extensions...
PTB-INFO: Installation of the PsychtoolboxKernelDriver is strongly recommended if you care about precise visual
PTB-INFO: onset timestamping or timing. See 'help PsychtoolboxKernelDriver' for installation instructions.
PTB-INFO: Fixed point precision integer framebuffer enabled.
PTB-INFO: System Frame buffer provides 8 bits for red channel.
PTB-INFO: System Frame buffer provides 8 bits for green channel.
PTB-INFO: System Frame buffer provides 8 bits for blue channel.
PTB-INFO: System frame buffer provides 8 bits for alpha channel, but effective alpha bits depends on imaging pipeline setup, if any.
PTB-DEBUG: PPM file magic is P6
 -> Ok
# Created by GIMP version 2.10.22 PNM plug-in
PTB-DEBUG: Recognized splash image of 1000 x 750 pixels, maxlevel 255. Loading...

OpenGL-Vendor / renderer / version are: Apple - Apple M1 - 2.1 Metal - 71.0.7

OpenGL-Extensions are: GL_ARB_color_buffer_float GL_ARB_depth_buffer_float GL_ARB_depth_clamp GL_ARB_depth_texture GL_ARB_draw_buffers GL_ARB_draw_elements_base_vertex GL_ARB_draw_instanced GL_ARB_fragment_program GL_ARB_fragment_program_shadow GL_ARB_fragment_shader GL_ARB_framebuffer_object GL_ARB_framebuffer_sRGB GL_ARB_half_float_pixel GL_ARB_half_float_vertex GL_ARB_imaging GL_ARB_instanced_arrays GL_ARB_multisample GL_ARB_multitexture GL_ARB_occlusion_query GL_ARB_pixel_buffer_object GL_ARB_point_parameters GL_ARB_point_sprite GL_ARB_provoking_vertex GL_ARB_seamless_cube_map GL_ARB_shader_objects GL_ARB_shader_texture_lod GL_ARB_shading_language_100 GL_ARB_shadow GL_ARB_shadow_ambient GL_ARB_sync GL_ARB_texture_border_clamp GL_ARB_texture_compression GL_ARB_texture_compression_rgtc GL_ARB_texture_cube_map GL_ARB_texture_env_add GL_ARB_texture_env_combine GL_ARB_texture_env_crossbar GL_ARB_texture_env_dot3 GL_ARB_texture_float GL_ARB_texture_mirrored_repeat GL_ARB_texture_non_power_of_two GL_ARB_texture_rectangle GL_ARB_texture_rg GL_ARB_transpose_matrix GL_ARB_vertex_array_bgra GL_ARB_vertex_blend GL_ARB_vertex_buffer_object GL_ARB_vertex_program GL_ARB_vertex_shader GL_ARB_window_pos GL_EXT_abgr GL_EXT_bgra GL_EXT_bindable_uniform GL_EXT_blend_color GL_EXT_blend_equation_separate GL_EXT_blend_func_separate GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_clip_volume_hint GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_draw_buffers2 GL_EXT_draw_range_elements GL_EXT_fog_coord GL_EXT_framebuffer_blit GL_EXT_framebuffer_multisample GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_framebuffer_object GL_EXT_framebuffer_sRGB GL_EXT_geometry_shader4 GL_EXT_gpu_program_parameters GL_EXT_gpu_shader4 GL_EXT_multi_draw_arrays GL_EXT_packed_depth_stencil GL_EXT_packed_float GL_EXT_provoking_vertex GL_EXT_rescale_normal GL_EXT_secondary_color GL_EXT_separate_specular_color GL_EXT_shadow_funcs GL_EXT_stencil_two_side GL_EXT_stencil_wrap GL_EXT_texture_array GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_s3tc GL_EXT_texture_env_add GL_EXT_texture_filter_anisotropic GL_EXT_texture_integer GL_EXT_texture_lod_bias GL_EXT_texture_rectangle GL_EXT_texture_shared_exponent GL_EXT_texture_sRGB GL_EXT_texture_sRGB_decode GL_EXT_timer_query GL_EXT_transform_feedback GL_EXT_vertex_array_bgra GL_APPLE_aux_depth_stencil GL_APPLE_client_storage GL_APPLE_element_array GL_APPLE_fence GL_APPLE_float_pixels GL_APPLE_flush_buffer_range GL_APPLE_flush_render GL_APPLE_packed_pixels GL_APPLE_pixel_buffer GL_APPLE_rgb_422 GL_APPLE_row_bytes GL_APPLE_specular_vector GL_APPLE_texture_range GL_APPLE_transform_hint GL_APPLE_vertex_array_object GL_APPLE_vertex_point_size GL_APPLE_vertex_program_evaluators GL_APPLE_ycbcr_422 GL_ATI_separate_stencil GL_ATI_texture_env_combine3 GL_ATI_texture_float GL_IBM_rasterpos_clip GL_NV_blend_square GL_NV_conditional_render GL_NV_depth_clamp GL_NV_fog_distance GL_NV_fragment_program_option GL_NV_fragment_program2 GL_NV_light_max_exponent GL_NV_texgen_reflection GL_NV_texture_barrier GL_NV_vertex_program2_option GL_NV_vertex_program3 GL_SGI_color_matrix GL_SGIS_generate_mipmap GL_SGIS_texture_edge_clamp GL_SGIS_texture_lod

PTB-DEBUG: Not running on Mesa graphics library.
PTB-DEBUG: Interrogating Low-level renderer capabilities for onscreen window with handle 10:
Indicator variables: FBO's 1, ATI_texture_float 1, ARB_texture_float 1, Vendor Apple, Renderer Apple M1.
Indicator variables: maxcolorattachments = 8, maxrectangletexturesize = 16384, maxnativealuinstructions = 262144.
GPU supports UYVY - YCrCb texture formats for optimized handling of video content.
GPU supports non-power-of-two textures.
Basic framebuffer objects with rectangle texture rendertargets supported --> RGBA8 rendertargets with blending.
Framebuffer objects support fast blitting between each other.
Framebuffer objects support anti-aliasing via multisampling.
Framebuffer objects support single-pass multisample resolve blits and image rescaling.
Hardware supports floating point textures of 16bpc and 32bpc float format.
Assuming hardware supports native OpenGL primitive smoothing (points, lines).
Float color value 0.5 -> fixed point reads back as 128 ==> Rounds.
No compiled in support for OpenML OML_sync_control extension.
PTB-DEBUG: Interrogation done.

PTB-DEBUG: glClear splash image top-left reference pixel: 90 0 90
PTB-INFO: Using OpenGL GL_TEXTURE_RECTANGLE_EXT extension for efficient high-performance texture mapping...
PTB-INFO: Threshold Settings for successfull video refresh calibration are: maxStdDev = 0.200000 msecs, maxDeviation = 10.000000 %, minSamples = 50, maxDuration = 5.000000 secs.


PTB-DEBUG: Output of all acquired samples of calibration run follows:
PTB-DEBUG: Sample 0: 0.000000
PTB-DEBUG: Sample 1: 0.031116
PTB-DEBUG: Sample 2: 0.018741
PTB-DEBUG: Sample 3: 0.005550
PTB-DEBUG: Sample 4: 0.014154
PTB-DEBUG: Sample 5: 0.019960
PTB-DEBUG: Sample 6: 0.012994
PTB-DEBUG: Sample 7: 0.016506
PTB-DEBUG: Sample 8: 0.016506
PTB-DEBUG: Sample 9: 0.016733
PTB-DEBUG: Sample 10: 0.017021
PTB-DEBUG: Sample 11: 0.015856
PTB-DEBUG: Sample 12: 0.016261
PTB-DEBUG: Sample 13: 0.016646
PTB-DEBUG: Sample 14: 0.016854
PTB-DEBUG: Sample 15: 0.017459
PTB-DEBUG: Sample 16: 0.016771
PTB-DEBUG: Sample 17: 0.016428
PTB-DEBUG: Sample 18: 0.016854
PTB-DEBUG: Sample 19: 0.016717
PTB-DEBUG: Sample 20: 0.016475
PTB-DEBUG: Sample 21: 0.016669
PTB-DEBUG: Sample 22: 0.016658
PTB-DEBUG: Sample 23: 0.016585
PTB-DEBUG: Sample 24: 0.016650
PTB-DEBUG: Sample 25: 0.016673
PTB-DEBUG: Sample 26: 0.016619
PTB-DEBUG: Sample 27: 0.016813
PTB-DEBUG: Sample 28: 0.016496
PTB-DEBUG: Sample 29: 0.016778
PTB-DEBUG: Sample 30: 0.016766
PTB-DEBUG: Sample 31: 0.016472
PTB-DEBUG: Sample 32: 0.016808
PTB-DEBUG: Sample 33: 0.016692
PTB-DEBUG: Sample 34: 0.016532
PTB-DEBUG: Sample 35: 0.016813
PTB-DEBUG: Sample 36: 0.016641
PTB-DEBUG: Sample 37: 0.016588
PTB-DEBUG: Sample 38: 0.016611
PTB-DEBUG: Sample 39: 0.016742
PTB-DEBUG: Sample 40: 0.015716
PTB-DEBUG: Sample 41: 0.016812
PTB-DEBUG: Sample 42: 0.017594
PTB-DEBUG: Sample 43: 0.016589
PTB-DEBUG: Sample 44: 0.016666
PTB-DEBUG: Sample 45: 0.016755
PTB-DEBUG: Sample 46: 0.016573
PTB-DEBUG: Sample 47: 0.016572
PTB-DEBUG: Sample 48: 0.016827
PTB-DEBUG: Sample 49: 0.016571
PTB-DEBUG: Sample 50: 0.016502
PTB-DEBUG: Sample 51: 0.016910
PTB-DEBUG: Sample 52: 0.016353
PTB-DEBUG: Sample 53: 0.016931
PTB-DEBUG: End of calibration data for this run...



PTB-INFO: OpenGL-Renderer is Apple :: Apple M1 :: 2.1 Metal - 71.0.7
PTB-INFO: Renderer has 10922 MB of VRAM and a maximum 10922 MB of texture memory.
PTB-INFO: VBL startline = 1800 , VBL Endline = -1
PTB-INFO: Beamposition queries unsupported or defective on this system. Using basic timestamping as fallback.
PTB-INFO: Timestamps returned by Screen('Flip') will be therefore less robust and accurate.
PTB-INFO: Measured monitor refresh interval from VBLsync = 16.718355 ms [59.814497 Hz]. (50 valid samples taken, stddev=0.720559 ms.)
PTB-INFO: Reported monitor refresh interval from operating system = 16.666667 ms [60.000000 Hz].
PTB-INFO: Small deviations between reported values are normal and no reason to worry.
PTB-WARNING: ==================================================================================================================
PTB-WARNING: DESKTOP COMPOSITOR IS ACTIVE! ALL FLIP STIMULUS ONSET TIMESTAMPS WILL BE VERY LIKELY UNRELIABLE AND LESS ACCURATE!
PTB-WARNING: STIMULUS ONSET TIMING WILL BE UNRELIABLE AS WELL, AND GRAPHICS PERFORMANCE MAY BE REDUCED!
PTB-WARNING: DO NOT USE THIS MODE FOR RUNNING REAL EXPERIMENT SESSIONS WITH ANY REQUIREMENTS FOR ACCURATE TIMING!
PTB-WARNING: ==================================================================================================================

WARNING: Couldn't compute a reliable estimate of monitor refresh interval! Trouble with VBL syncing?!?

----- ! PTB - ERROR: SYNCHRONIZATION FAILURE ! -----

One or more internal checks (see Warnings above) indicate that synchronization
of Psychtoolbox to the vertical retrace (VBL) is not working on your setup.

This will seriously impair proper stimulus presentation and stimulus presentation timing!
Please read 'help SyncTrouble' for information about how to solve or work-around the problem.
You can force Psychtoolbox to continue, despite the severe problems, by adding the command
Screen('Preference', 'SkipSyncTests', 1); at the top of your script, if you really know what you are doing.


PTB-INFO: Trying to enable my builtin panel-fitter on user request.
PTB-INFO: Psychtoolbox imaging pipeline starting up for window with requested imagingmode 33793 ...
PTB-INFO: Will use 8 bits per color component framebuffer for stimulus drawing.
PTB-INFO: Enabling panel fitter. Providing virtual framebuffer of 1440 x 900 pixels size.
PTB-DEBUG: Only colorbuffer texture attached to FBO, no depth- or stencil buffers requested...
PTB-DEBUG: FBO has 8 bits precision per color component in fixed point format coltexid 1 with 0 depths buffer bits and 0 stencil buffer bits.
PTB-INFO: Will use 8 bits per color component framebuffer for stimulus post-processing (if any).
PTB-DEBUG: Buffer mappings follow...
fboCount = 2
finalizedFBO = 0, 0
preConversionFBO = 0, -1, -1, 0
processedDrawBufferFBO = 0 -1 -1
inputBufferFBO = 0 -1 
drawBufferFBO = 1 -1 

fboTable content:
fboTable[0]: textarget 34037 : coltexid 0 : format 0 : MSAA 0 : width 2880 x height 1800
fboTable[1]: textarget 34037 : coltexid 1 : format 32856 : MSAA 0 : width 1440 x height 900
-------------------------------------

PTB-DEBUG: Panel-Fitter scaled Scaleblit: [0 0 1440 900] -> [0 0 2880 1800], Rotation=0, RotCenter=[1440, 900]
PTB-DEBUG: Panel-Fitter scaled Scaleblit: [0 0 1440 900] -> [0 0 2880 1800], Rotation=0, RotCenter=[1440, 900]
PTB-DEBUG: Panel-Fitter scaled Scaleblit: [0 0 1440 900] -> [0 0 2880 1800], Rotation=0, RotCenter=[1440, 900]
PTB-INFO: Cocoa + Retina scaling. Scaling factor is 2.000000x.
PTB-DEBUG: Panel-Fitter scaled Scaleblit: [0 0 1440 900] -> [0 0 2880 1800], Rotation=0, RotCenter=[1440, 900]
PTB-DEBUG: Panel-Fitter scaled Scaleblit: [0 0 1440 900] -> [0 0 2880 1800], Rotation=0, RotCenter=[1440, 900]
PTB-DEBUG: Swaprequest too close to last swap vbl (0.000217 secs) or between forbidden scanline 1 and 50. Delaying...
PTB-DEBUG: Panel-Fitter scaled Scaleblit: [0 0 1440 900] -> [0 0 2880 1800], Rotation=0, RotCenter=[1440, 900]
PTB-DEBUG: Swaprequest too close to last swap vbl (0.000144 secs) or between forbidden scanline 1 and 50. Delaying...

ans =

10

Yep, timing is about as broken as it can be. Pretty hopeless.

Seems it detected the keyboard correctly.

What’s the output of d = PsychHID('Devices')? The full contents of the struct array, all the gory details.

Does an external standard keyboard work?

Did the system ask you or permission to allow Matlab to access input, as it does on macOS 10.14 and later? Ie. did that dialog box pop up and you "Allow"ed it? Maybe double-check in the macOS settings → Privacy & Security → Some stuff related to input that Matlab is listed there as permitted to use low-level input.

-mario

According to Security & Privacy settings, the only thing MATLAB has requested access to is “Input Monitoring”, and that is enabled. Yes, there was initially a popup for that.

An external keyboard doesn’t work either – even when calling KbStrokeWait() with the appropriate index.

Output from PsychHID('Devices') below.

>> d = PsychHID('Devices');
>> for i = 1:length(d)
>> disp(d(i))
>> end
     usagePageValue: 65280
         usageValue: 255
          usageName: 'Page: 0xff00, Usage: 0xff'
              index: 1
          transport: 'SPU'
           vendorID: 0
          productID: 0
            version: 0
       manufacturer: 'Apple'
            product: ''
       serialNumber: ''
         locationID: 0
        interfaceID: -1
      totalElements: 0
           features: 0
             inputs: 0
            outputs: 0
        collections: 1
               axes: 0
            buttons: 0
               hats: 0
            sliders: 0
              dials: 0
             wheels: 0
    touchDeviceType: -1
     maxTouchpoints: -1

 usagePageValue: 65280
     usageValue: 4
      usageName: 'Page: 0xff00, Usage: 0x4'
          index: 2
      transport: 'SPU'
       vendorID: 0
      productID: 0
        version: 0
   manufacturer: 'Apple'
        product: ''
   serialNumber: ''
     locationID: 0
    interfaceID: -1
  totalElements: 0
       features: 0
         inputs: 0
        outputs: 0
    collections: 1
           axes: 0
        buttons: 0
           hats: 0
        sliders: 0
          dials: 0
         wheels: 0
touchDeviceType: -1
 maxTouchpoints: -1

 usagePageValue: 65280
     usageValue: 72
      usageName: 'Page: 0xff00, Usage: 0x48'
          index: 3
      transport: 'SPMI'
       vendorID: 0
      productID: 0
        version: 0
   manufacturer: 'APPL'
        product: 'BTM'
   serialNumber: ''
     locationID: 0
    interfaceID: -1
  totalElements: 4
       features: 3
         inputs: 1
        outputs: 0
    collections: 2
           axes: 0
        buttons: 0
           hats: 0
        sliders: 0
          dials: 0
         wheels: 0
touchDeviceType: -1
 maxTouchpoints: -1

 usagePageValue: 12
     usageValue: 1
      usageName: 'Consumer Usage 0x1'
          index: 4
      transport: 'Audio'
       vendorID: 0
      productID: 0
        version: 0
   manufacturer: 'Apple'
        product: 'Headset'
   serialNumber: ''
     locationID: 0
    interfaceID: -1
  totalElements: 3
       features: 0
         inputs: 3
        outputs: 0
    collections: 1
           axes: 0
        buttons: 0
           hats: 0
        sliders: 0
          dials: 0
         wheels: 0
touchDeviceType: -1
 maxTouchpoints: -1

 usagePageValue: 65280
     usageValue: 15
      usageName: 'Page: 0xff00, Usage: 0xf'
          index: 5
      transport: 'USB'
       vendorID: 1452
      productID: 641
        version: 0
   manufacturer: ''
        product: 'Keyboard Backlight'
   serialNumber: ''
     locationID: 0
    interfaceID: -1
  totalElements: 4
       features: 4
         inputs: 0
        outputs: 0
    collections: 1
           axes: 0
        buttons: 0
           hats: 0
        sliders: 0
          dials: 0
         wheels: 0
touchDeviceType: -1
 maxTouchpoints: -1

 usagePageValue: 1
     usageValue: 2
      usageName: 'Mouse'
          index: 6
      transport: 'SPI'
       vendorID: 1452
      productID: 641
        version: 2357
   manufacturer: 'Apple Inc.'
        product: 'Apple Internal Keyboard / Trackpad'
   serialNumber: <REMOVED>
     locationID: 34
    interfaceID: -1
  totalElements: 1207
       features: 0
         inputs: 1207
        outputs: 0
    collections: 4
           axes: 2
        buttons: 3
           hats: 0
        sliders: 0
          dials: 0
         wheels: 0
touchDeviceType: -1
 maxTouchpoints: -1

 usagePageValue: 65280
     usageValue: 3
      usageName: 'Page: 0xff00, Usage: 0x3'
          index: 7
      transport: 'SPI'
       vendorID: 1452
      productID: 641
        version: 2357
   manufacturer: 'Apple Inc.'
        product: 'Apple Internal Keyboard / Trackpad'
   serialNumber: <REMOVED>
     locationID: 34
    interfaceID: -1
  totalElements: 1
       features: 0
         inputs: 1
        outputs: 0
    collections: 1
           axes: 0
        buttons: 0
           hats: 0
        sliders: 0
          dials: 0
         wheels: 0
touchDeviceType: -1
 maxTouchpoints: -1

 usagePageValue: 1
     usageValue: 6
      usageName: 'Keyboard'
          index: 8
      transport: 'SPI'
       vendorID: 1452
      productID: 641
        version: 2357
   manufacturer: 'Apple Inc.'
        product: 'Apple Internal Keyboard / Trackpad'
   serialNumber: <REMOVED>
     locationID: 34
    interfaceID: -1
  totalElements: 285
       features: 1
         inputs: 279
        outputs: 5
    collections: 3
           axes: 0
        buttons: 0
           hats: 0
        sliders: 0
          dials: 0
         wheels: 0
touchDeviceType: -1
 maxTouchpoints: -1

 usagePageValue: 65280
     usageValue: 13
      usageName: 'Page: 0xff00, Usage: 0xd'
          index: 9
      transport: 'SPI'
       vendorID: 1452
      productID: 641
        version: 2357
   manufacturer: 'Apple Inc.'
        product: 'Apple Internal Keyboard / Trackpad'
   serialNumber: <REMOVED>
     locationID: 34
    interfaceID: -1
  totalElements: 2
       features: 0
         inputs: 1
        outputs: 1
    collections: 1
           axes: 0
        buttons: 0
           hats: 0
        sliders: 0
          dials: 0
         wheels: 0
touchDeviceType: -1
 maxTouchpoints: -1

 usagePageValue: 65280
     usageValue: 11
      usageName: 'Page: 0xff00, Usage: 0xb'
          index: 10
      transport: 'SPI'
       vendorID: 1452
      productID: 641
        version: 2357
   manufacturer: 'Apple Inc.'
        product: 'Apple Internal Keyboard / Trackpad'
   serialNumber: <REMOVED>
     locationID: 34
    interfaceID: -1
  totalElements: 1
       features: 0
         inputs: 1
        outputs: 0
    collections: 1
           axes: 0
        buttons: 0
           hats: 0
        sliders: 0
          dials: 0
         wheels: 0
touchDeviceType: -1
 maxTouchpoints: -1

Ok, that’s what you need. So Apples ideas of security are probably not getting in the way this time, at least not in an obvious way.

Ok, there is afaics only one true keyboard device in that list, which does get properly selected, so keyboard enumeration/detection/selection does not seem to be a problem either.

I guess this means we are pretty much done here, as this has taken up more than enough unpaid work time of myself, multiple hours lost by now. This doesn’t look like an easy and quick fix, more like the usual “down the rabbit hole” of dealing with Apples abominations of software engineering. I think we do have some users on macOS 11 + IntelMac which did not report keyboard trouble, and i assume the external keyboard works otherwise fine on other Macs, so these are new M1 specific screwups.

If you want, you can check if KbQueueDemo works any better - i doubt it. Or call setenv('PSYCHHID_TELLME', '1'); and then run KbQueueCreate for some debug output.

Or call the regular KbCheck(-GetKeyboardIndices()), passing in the negated value of the keyboard index, to output some low-level debug output.
Maybe it helps at some point to diagnose further, once somebody commits to pay all the required work time.

I will put a warning on our website that M1 is completely unsupported with PTB for the foreseeable future. But as we learned in this thread, installing Ubuntu Linux for ARM / RaspberryPi in a virtual machine seems to be a much better working solution for toy applications and learning, so Linux to the rescue…
Post for reference: Using toolbox with Big Sur and M1 MacBook? - #4 by msamy

-mario

As if by magic…

Big Sur offered an OS update today. Ordinarily I wouldn’t download updates until they are fully supported by PTB, but since I’m on a brand new MacBook which is already ‘broken’, I figured there’s nothing to lose by downloading it. And, just like that… keyboard functions are working again in PTB. So, it looks like it was an Apple screwup, but this time around they’ve fixed it themselves.

Not as much luck with stimulus timing or being able to disable the ConserveVRAM workaround, but the point is that with the latest update it’s now possible to run PTB on an M1 for basic debugging.

Hello there!
Sine the release of the Mac Big Sur Update I recognised similar problems using PTB. Glad to see that the current update might at least fix the bug of the KBXXX functions.
Just a sidenote on that: During my research for several solutions, I realised that PTB had a similar issue with High Sierra. As far as I know, the bug was caused by the new touchbar (they called it “TouchBar Gimmik”: Psychtoolbox-3 - Psychtoolbox News).

I recognized a new bug that seems to go deeper.
When I connect a projector with 4:3 aspect ratio to the new MacBook via USB-C, the Macbook does not recognise the resolution and projects something else in 16:9 on PTB. The Screen(‘Resolution’, 800, 600) function does not help to display the stimulus full screen accordingly. However, I can set the size of the window to 800x600 without any problems. I know that this is primarily a problem of the Macbook and the connections, yet it continues:
If I use SwitchResX to change the resolution on the Mac to 800x600 or any other size I want (which works fine), and run the script so that the stimulus appears full screen, then the stimulus appears 16:9 and also scaled down (font sizes and point sizes appear smaller). When the script is finished, I see on the Mac the resolution previously set with SwitchResX.
So there is a bug in some circuitry. Because as soon as I want to display something full screen, the resolution jumps back to 16:9. The data collected then is of course useless.
It is important to note that the PTB-window must be full screen and the bug appears without any devices connected.
Besides that, after trying out different resolutions, other bugs appear. When I use Screen(‘DrawLine’) to move a line on the PTB screen with the mouse, the line does not reach the right edge of the screen completely. Instead, I only reach ~70% of the screen. This bug disappears again at some point, so I have not yet been able to identify a systematic approach to this.
In the meantime I draw everything directly into the OpenGL world to avoid errors caused by the wrong aspect ratios. Also, I can always fall back on a Windows PC that doesn’t have all these bugs. Still, the resolution in full screen mode on Mac Big Sur doesn’t seem to be able to be changed. Have you noticed this problem? Do you know a solution? I have not yet updated my operating system to 11.3. I will inform you if any of my reported bugs may also disappear by the OS update.

I’ve not noticed this behaviour, but am not using an external screen (I only use PTB on Mac to do basic debugging of scripts at home; in the lab we use low-latency Linux to avoid all the timing problems).

Adding the following line can sometimes deal with stimuli that appear unexpectedly small (by accounting for Mac’s ‘retina display’ feature). Although, I don’t think this does anything to fix aspect ratio issues, and if you use this line then you need to be careful to make sure you are rendering stimuli at the size you expect.

PsychImaging('AddTask','General','UseRetinaResolution')

Thanks for your response!
I noticed as well rendering issues. For instance, I have a skript that simulates motion through an OpenGL world. Your task can be to move a line (Screen(‘DrawLine’) function) to the place where you head. Through extensive testing, I concluded that the aspect ratio of the OpenGL world is always 4:3, while the PTB-screen on which I draw the line changes depending on what you either manually set using PsychImaging(‘OpenWindow’)-function, or what the mac sets for you when you open your PTB-window as full screen. This problem is new because older macs were able to either detect the correct ratio of the connected screen, or switched to 4:3 when calling Screen(‘Resolution’). I solved this bug by drawing the line directly in the OpenGL world. However, the ratio issue of PTB-screen still exists.

Using your suggestion did not help me but I still appreciate your advice.

Update on this reported bug:

Besides that, after trying out different resolutions, other bugs appear. When I use Screen(‘DrawLine’) to move a line on the PTB screen with the mouse, the line does not reach the right edge of the screen completely. Instead, I only reach ~70% of the screen. This bug disappears again at some point, so I have not yet been able to identify a systematic approach to this.

While testing your PsychImaging function with various resolution, I noticed an interesting interaction effect. When using HiDPI resolution, the stimulius appears normal but I can only move the line in a reduced area of the screen. Again, the objects in my OpenGL world are not affected by this bug. Using the same resolution but not as HiDPi however inverts the pattern. So the stimulus appears unexpectedly small (despite your suggested code) but I can move the line on the PTB-screen properly.

I found a mode where I could scale the aspect ratio properly. When I connect my projector (via USB-C) and use SwitchResX to select the correct resolution for the projector (in this case 800:600 with 75Hz), and both screens (Macbook and projector) are synchronised, I finally get the resolution and rendering as desired. I didn’t expect the stimulus on full screen to be displayed correctly with a connected device. Anyway, there seems to be at least one workaround for the ratio issue for Big Sur.

Just wanted to let you know this. Hope this helps others as well.

As this all relates to the general brokenness of Apple macOS, and not specifically to the additional brokenness of Apple macOS on the new M1 machines, it would be good to not side-track by posting these problems under this thread.

Some remarks:

  • macOS 11 is not officially supported or tested at all with current PTB, you are on your own if you use it.

  • Using “Mirror mode” will likely break timing and timestamping, i would not recommend it if timing matters at all. But then of course i wouldn’t recommend using Apples toys in the first place if timing, precision or control matters.

  • The resolution switching and mouse problems can be related to the hacks that PTB has to employ to work around a pile of macOS bugs, in order to get ok timing at least for regular fullscreen windows, and with the PsychtoolboxKernelDriver properly installed on supported hardwar. It is surprising that ‘UseRetinaResolution’ would do anything on a projector that is clearly not HiDPI. On Retina displays it can help to avoid mismatched mouse coordinate reporting wrt. screen coordinates. Maybe it also helps on mixed DPI multi-display setups. There are known problems with this if not using Retina native resolution, thousands of euros of so far unpaid work time have been wasted not managing to fix this for all cases, so no further likely futile unpaid attempts will be made from my side to improve this.

Cool, and just like that we lost a few more hundred Euros of worktime due to Apple induced issues, while the iToys company reports record earnings for last quarter. Seems like the worse of a job they do wrt. software and hardware quality, design and treatment of developers and pro customers, the more they get rewarded by consumers buying their toys like crazy.

I don’t think disabling that workaround will ever be possible on M1. For M1 and presumably future Apple machines with Apple proprietary gpu’s, the Apple proprietary OpenGL driver is not a real native OpenGL driver anymore, but an OpenGL-on-Metal emulator, which translates all OpenGL calls into calls for Apples proprietary Metal graphics api (Hence the word “Metal” in PTB’s renderer information). Apple Metal can only display rendered content onscreen by rendering into a CoreAnimation CAMetalLayer, attached to a layer backed NSWindow/NSView. It can’t use the low-level CGL OpenGL fullscreen display api’s we’d normally use (CGLSetFullScreenOnDisplay etc.), because it is not an OpenGL renderer. So all our current tricks for proper timing are dead. But even if that would work by miracle (= Apple deciding to completely redesign/rewrite their OpenGL drivers for M1 from scratch – pretty unlikely), your debug output indicated that various other critically required pieces needed for the timing hacks also got broken by Apple for the M1. And proper timing through the workaround which uses NSWindow/NSView/NSOpenGL got broken by Apple since many years, probably with the breakage already starting around 2012 for macOS 10.7 on NVidia, although it got much worse around 10.11 or 10.12, affecting NVidia and many AMD’s and completely kicked the bucket with 10.14, killing timing all gpu’s supported by macOS. The current pile of hacks on top of hacks on top of hacks to make things work in many cases (but by far not all of them) on macOS 10 took close to 500 hours to figure out, with many false starts over the years, most of the time (except for 100 hours) being unpaid by the community, as usual. So this is all dead now with macOS 11 and the M1 gpu’s, with little chance of resurrection ever.

What we do have since the latest PTB beta release is a Vulkan based prototype implementation to explore the viability of using Metal as a display backend via the open-source Khronos MoltenVK Vulkan-on-Metal graphics driver. Ie. render and post-process visual stimuli as usual via Apples proprietary OpenGL implementation → submit final stimulus image to Vulkan for display by using our relatively new PsychVulkan driver → MoltenVK driver translates Vulkan → Metal for display via Metal, Apples proprietary Metal displays the image.

According to Apple propaganda, Metal is the greatest invention since sliced bread and the one true way to do graphics on macOS. According to my results after just spending 174 utterly depressing hours of work, trying to get Metal to behave even half-way reasonable wrt. presentation timing and timestamping, my assessment so far is that this is even a bigger dumpster fire than Apples OpenGL implementation when it comes to stimulus timing and timestamping. A lot of time was spent making sure the failures i see are not caused by any bugs or limitations in PTB, or the open-source MoltenVK Vulkan-to-Metal driver, or of the general approach which works nicely on both Linux and Windows-10, so these are again macOS bugs and problems courtesy of Apple, only fixable by Apple, with no known workarounds. My tests are limited to a MacBookPro 2017 running macOS 10.15.7 with both AMD and Intel graphics so far. Who knows, maybe macOS 11 on M1 is less broken in that area than macOS 10 – although that would be the first time anything would be better on the dumpster fire that Big Sur seems to be, even compared to the temple of sadness that was recent macOS 10?

But for Apple M1 and its successors, i assume it will be either this Vulkan/Metal approach, or nothing, if timing matters at all. There are no more tricks or workarounds left from my side and Apple pretty much closed any opportunities for developing new workarounds in the future, given their proprietary gpu, and the way the locked down macOS 11 M1, and given they announced already their intention to lock it down even more in future hardware and OS versions.

This new experimental code is essentially unsupported by myself in case of problems. It has various known limitations, e.g., it only supports most basic use of Screen('Flip', window, when), not any more advanced PTB mechanisms like async flips, frame-sequential stereo, non-vsynced updates etc. Any future improvements to it will have to be paid by contracting us. Guidance wrt. use of the current implementation will also require priority support, so my time is paid for and the costs of failure on other machine + OS versions are not on our side. Some demos are Vulkan enabled for testing if an optional flag is specified, e.g., PerceptualVBLSyncTest and VBLSyncTest for basic timing checks. SimpleHDRDemo is another one.

-mario