Thanks for your response!
I noticed as well rendering issues. For instance, I have a skript that simulates motion through an OpenGL world. Your task can be to move a line (Screen(‘DrawLine’) function) to the place where you head. Through extensive testing, I concluded that the aspect ratio of the OpenGL world is always 4:3, while the PTB-screen on which I draw the line changes depending on what you either manually set using PsychImaging(‘OpenWindow’)-function, or what the mac sets for you when you open your PTB-window as full screen. This problem is new because older macs were able to either detect the correct ratio of the connected screen, or switched to 4:3 when calling Screen(‘Resolution’). I solved this bug by drawing the line directly in the OpenGL world. However, the ratio issue of PTB-screen still exists.
Using your suggestion did not help me but I still appreciate your advice.
Update on this reported bug:
Besides that, after trying out different resolutions, other bugs appear. When I use Screen(‘DrawLine’) to move a line on the PTB screen with the mouse, the line does not reach the right edge of the screen completely. Instead, I only reach ~70% of the screen. This bug disappears again at some point, so I have not yet been able to identify a systematic approach to this.
While testing your PsychImaging function with various resolution, I noticed an interesting interaction effect. When using HiDPI resolution, the stimulius appears normal but I can only move the line in a reduced area of the screen. Again, the objects in my OpenGL world are not affected by this bug. Using the same resolution but not as HiDPi however inverts the pattern. So the stimulus appears unexpectedly small (despite your suggested code) but I can move the line on the PTB-screen properly.
I found a mode where I could scale the aspect ratio properly. When I connect my projector (via USB-C) and use SwitchResX to select the correct resolution for the projector (in this case 800:600 with 75Hz), and both screens (Macbook and projector) are synchronised, I finally get the resolution and rendering as desired. I didn’t expect the stimulus on full screen to be displayed correctly with a connected device. Anyway, there seems to be at least one workaround for the ratio issue for Big Sur.
Just wanted to let you know this. Hope this helps others as well.
As this all relates to the general brokenness of Apple macOS, and not specifically to the additional brokenness of Apple macOS on the new M1 machines, it would be good to not side-track by posting these problems under this thread.
Some remarks:
macOS 11 is not officially supported or tested at all with current PTB, you are on your own if you use it.
Using “Mirror mode” will likely break timing and timestamping, i would not recommend it if timing matters at all. But then of course i wouldn’t recommend using Apples toys in the first place if timing, precision or control matters.
The resolution switching and mouse problems can be related to the hacks that PTB has to employ to work around a pile of macOS bugs, in order to get ok timing at least for regular fullscreen windows, and with the PsychtoolboxKernelDriver properly installed on supported hardwar. It is surprising that ‘UseRetinaResolution’ would do anything on a projector that is clearly not HiDPI. On Retina displays it can help to avoid mismatched mouse coordinate reporting wrt. screen coordinates. Maybe it also helps on mixed DPI multi-display setups. There are known problems with this if not using Retina native resolution, thousands of euros of so far unpaid work time have been wasted not managing to fix this for all cases, so no further likely futile unpaid attempts will be made from my side to improve this.
Cool, and just like that we lost a few more hundred Euros of worktime due to Apple induced issues, while the iToys company reports record earnings for last quarter. Seems like the worse of a job they do wrt. software and hardware quality, design and treatment of developers and pro customers, the more they get rewarded by consumers buying their toys like crazy.
I don’t think disabling that workaround will ever be possible on M1. For M1 and presumably future Apple machines with Apple proprietary gpu’s, the Apple proprietary OpenGL driver is not a real native OpenGL driver anymore, but an OpenGL-on-Metal emulator, which translates all OpenGL calls into calls for Apples proprietary Metal graphics api (Hence the word “Metal” in PTB’s renderer information). Apple Metal can only display rendered content onscreen by rendering into a CoreAnimation CAMetalLayer, attached to a layer backed NSWindow/NSView. It can’t use the low-level CGL OpenGL fullscreen display api’s we’d normally use (CGLSetFullScreenOnDisplay etc.), because it is not an OpenGL renderer. So all our current tricks for proper timing are dead. But even if that would work by miracle (= Apple deciding to completely redesign/rewrite their OpenGL drivers for M1 from scratch – pretty unlikely), your debug output indicated that various other critically required pieces needed for the timing hacks also got broken by Apple for the M1. And proper timing through the workaround which uses NSWindow/NSView/NSOpenGL got broken by Apple since many years, probably with the breakage already starting around 2012 for macOS 10.7 on NVidia, although it got much worse around 10.11 or 10.12, affecting NVidia and many AMD’s and completely kicked the bucket with 10.14, killing timing all gpu’s supported by macOS. The current pile of hacks on top of hacks on top of hacks to make things work in many cases (but by far not all of them) on macOS 10 took close to 500 hours to figure out, with many false starts over the years, most of the time (except for 100 hours) being unpaid by the community, as usual. So this is all dead now with macOS 11 and the M1 gpu’s, with little chance of resurrection ever.
What we do have since the latest PTB beta release is a Vulkan based prototype implementation to explore the viability of using Metal as a display backend via the open-source Khronos MoltenVK Vulkan-on-Metal graphics driver. Ie. render and post-process visual stimuli as usual via Apples proprietary OpenGL implementation → submit final stimulus image to Vulkan for display by using our relatively new PsychVulkan driver → MoltenVK driver translates Vulkan → Metal for display via Metal, Apples proprietary Metal displays the image.
According to Apple propaganda, Metal is the greatest invention since sliced bread and the one true way to do graphics on macOS. According to my results after just spending 174 utterly depressing hours of work, trying to get Metal to behave even half-way reasonable wrt. presentation timing and timestamping, my assessment so far is that this is even a bigger dumpster fire than Apples OpenGL implementation when it comes to stimulus timing and timestamping. A lot of time was spent making sure the failures i see are not caused by any bugs or limitations in PTB, or the open-source MoltenVK Vulkan-to-Metal driver, or of the general approach which works nicely on both Linux and Windows-10, so these are again macOS bugs and problems courtesy of Apple, only fixable by Apple, with no known workarounds. My tests are limited to a MacBookPro 2017 running macOS 10.15.7 with both AMD and Intel graphics so far. Who knows, maybe macOS 11 on M1 is less broken in that area than macOS 10 – although that would be the first time anything would be better on the dumpster fire that Big Sur seems to be, even compared to the temple of sadness that was recent macOS 10?
But for Apple M1 and its successors, i assume it will be either this Vulkan/Metal approach, or nothing, if timing matters at all. There are no more tricks or workarounds left from my side and Apple pretty much closed any opportunities for developing new workarounds in the future, given their proprietary gpu, and the way the locked down macOS 11 M1, and given they announced already their intention to lock it down even more in future hardware and OS versions.
This new experimental code is essentially unsupported by myself in case of problems. It has various known limitations, e.g., it only supports most basic use of Screen('Flip', window, when), not any more advanced PTB mechanisms like async flips, frame-sequential stereo, non-vsynced updates etc. Any future improvements to it will have to be paid by contracting us. Guidance wrt. use of the current implementation will also require priority support, so my time is paid for and the costs of failure on other machine + OS versions are not on our side. Some demos are Vulkan enabled for testing if an optional flag is specified, e.g., PerceptualVBLSyncTest and VBLSyncTest for basic timing checks. SimpleHDRDemo is another one.
Just asked the Mathworks / INCF organizers, and apparently nope, not possible. Maybe next year.
In any case, we will upload Celia’s talk to Youtube, once the meeting is over. And the whole thing is recorded, and presumably uploaded at some point somewhere. Although last time in 2022 it took more than a year for them to manage, and not all content, sadly not the Q & A, was included.
Peter, we will mention/credit you as one of the unpaid Team Psychtoolbox volunteer members btw., for your great work at the tutorials and demos.
As always, very happy to be providing some support to PTB through the demos. It is basically the main thing students recognised me for at conferences. So it is great to have been of use to folk learning programming. I might finally publish a paper on them this year, but I have been saying that for many years. Stuff just gets in the way.
As always, thanks for all your dedicated work on PTB. It is an utterly fantastic programming system for research. Many fields of research owe you a great debt. You should get a paper out. AFAIK we can only cite you via a poster?
Given the VSS recording might take a while to get up. Are you able to state the news you have hinted at r.e. ARM support. I see recently over the last few days Windows are making another ARM push.
Hi Peter, I attended the session. In regards to the Psychtoolbox, the gist is that Mario is the main developer, this is his full time job, but the voluntary contributions to support PTB are given by a very tiny fraction of users despite various attempts and ideas in the last 10 years, and this has just become unsustainable. So in the future, users will need to purchase an annual license to use the mex files for the Apple Silicon version of PTB. The source code is still publicly available if you want to compile it yourself (you will like find this not to be worth your time), and I believe that for now the Linux Mex files will still be free.
There seemed to be general support among the audience for paying a license fee. In fact, probably Mario has been the biggest opponent of it over the years, because he strongly supports the open source philosophy. But also he has spent an inordinate amount of time specifically getting PTB to work on the Mac, and there is still a lot more work to be done, and PTB also requires constant maintenance.
The nature of the license fee has not been determined, but probably there will be single machine options, a lab license for like 5 computers (something like €200/year?), and perhaps a site license. Mario said that they would like feedback and suggestions about the licensing options.
There were some other presentations about other Matlab toolboxes, in particular PsychBench, which is a front end, so to speak, for building experiments with PTB without needing so much programming. You can build an experiment using some high-level functions in Matlab, or even using a table of functions in a spreadsheet software.
There was one other tool, I forget the name, where they are also developing for MacOS, in particular the Metal library, which is necessary for high precision timing. So perhaps Mario and that developer can share some effort with respects to that. Metal is Mac-specific, in contrast to OpenGL or whatever is used for Linux and Windows, another reason why the Apple Silicon version requires so much extra work.
One other thing, PTB is officially supported by Mathworks and is/will be available to be installed via their website/interface. (Mario also maintains compatibility with Octave.)
Anyone else who attended, or Mario, feel free to correct anything I wrote above.
So the only license fee for PTB is specifically for only macOS on ARM? Is there any donated money for this macOS on ARM development work which would now be behind a paywall? Linux is free, but what about Windows? Basically, is the model “macOS ARM users fund the other platforms”?
Will Windows and Linux on ARM also be behind a paywall when they come?
I guess the user base is still primarily Windows and macOS, so that would be the place to target for revenue. But I am not sure why it is focused on macOS ARM users.
Possibly parsed things incorrectly.
I think the Metal based library was MGL?
Regardless. Very glad to see PTB moving to a more sustainable footing.
Maybe a better way to think about it is that now Apple Silicon users are allowed to get licenses for PTB, but the other platforms aren’t yet. We couldn’t really charge for existing versions, because maybe people wouldn’t ever update, etc. But the only way to get the new Apple Mex files is to get a license. Hopefully this will encourage people to get a license to support the development. Also, it took considerable effort (I think Mario said 366 hours or so) to get the new version running, which was not really on Mario’s “to do” list, and which was only very partially funded (I and someone else paid for 50 hours each, not sure what else).
Presumably at some point the other platforms might be able to have licenses too, it just makes the most sense to start it with the new Mac version. Note that you could still use the Intel versions on your Apple Silicon Mac, you just wouldn’t have the native version, and of course the timing won’t be precise.
Yes, I do not see this working if it were only macOS on ARM users who were required to get licences. I would assume in terms of revenue Windows is bar far the biggest user base, followed by macOS then Linux (though I do not have numbers, so that is a guess). I could see a model with Windows and macOS supporting Linux which remained free.
However, that would also have to be reflected in development. If Windows and macOS user fund the model, that should be reflected in the development priorities. For completely understandable reasons Linux has been the focus for the last while.
I hope that this also stops the aggressive “upgrade to Linux” / you cannot possibly do good science on anything other than Linux, mantra. Whilst I fully understand the superiorities of Linux for some things, that simply does not reflect the day to day reality of labs, or their needs.
I might be biased, but I consider PTB a fantastic system for all platforms. Plus alienating the majority of your user base (even if for good intentions) is just a bad business model.
Anyhow, I think this is fantastic news and I hope this process puts PTB on a firm footing going forwards!
Is there any timeline on when licences can be purchased? Also, information on site / group licenses would be good. We use PTB in teaching and our students 99.9% use Windows and macOS.
Great VSS talk @celiafoster (@mariokleiner@dee). I was going to comment on that thread but the thread seems locked? So here is the next best.
I there any timescale for the macOS ARM support (it was described as “prototype” but appearing “soon”).
Also any further details of the funding structure? In particular group / site licenses. I use PTB for teaching, but with the existing “donation” model it was not possible to contribute funding through the university. Despite much effort.
Finally… Any guidance of how the funding model will work going forwards (see previous post).
Many thanks. Awesome that PTB funding and sustainability is moving forwards.
Once we have the whole “pay-to-play” license model in place, ie. a new online shop, infrastructure, license checking in PTB etc. I can release a first iteration. So the question is mostly how long it will take us to implement the new license model.
The current code works reasonably well, at least as good as what you can use now with Octave/Matlab for Intel under Rosetta2 emulation, just with native Octave for ARM or native Matlab R2023b+ for ARM, with no obvious problems. So it should be quite a step up in ease of installation / usability / future proofing etc.
So far the work done since 2020 involved:
~1000 work hours worth of work, funded by a big Linux (+ a bit Windows) contract multiple years ago, which was not aimed at macOS at all, as the contract givers consider macOS something not worth supporting at all. Their primary priority was Linux, followed by Windows as a nice to have extra. Luckily the knowledge gained during that work, and infrastructure implemented, proved to be very useful and to a large part recyclable / transferable for a potential solution to macOS on Apple Silicon Macs. So these crucial 1000 hours are a free gift from the Linux side of things to the macOS side of things. Without Linux and my work there, none of this macOS work would be possible, as the macOS side of things lives a lot from technology and knowledge transfer from Linux. As does the Windows side of things, at least as far as knowledge transfer is concerned.
~ 70 work hours of work sponsored by Mathworks in 2021 or 2022, can’t remember, for some proof-of-concept prototype port of the Linux+Windows technology to macOS for IntelMacs and some basic feasibility testing.
~ 200 hours of completely unpaid and unfunded work by myself, to move that forward, because despite my justified low opinion of Apple macOS as a system for experiment data collection, I still have more empathy for Apple users than what I’m given credit for, and my inner geek also wanted to know if one can “make pigs fly”.
Based on that, @keith funded 50 hours of work this year, on top of which I gifted another 13 hours of free work, to bring Apple Silicon support to its current state.
So the current state reflects ~1000 hours of Linux work repurposed/translated to macOS, and ~333 hours of dedicated work on macOS for Apple Silicon enablement.
Basic tests, e.g., of timing, beyond that “basic work” look promising, but I haven’t had time yet to perform the extensive kind of testing I normally do, e.g., involving hardware testing equipment.
We have potential upcoming funding for another 50 work hours, but this is not yet set in stone and still could go sideways. It would fund more extensive tests, needed refinements and whatever may become necessary based on test results.
It was meant as a donation model in disguise, but mostly designed as also providing a real service of value by itself, so universities should not have had any or many issues with it, based on feedback of hundreds of P.I’s. Their feedback was wrong, and the fact it essentially didn’t work drives the point home that nothing in the “carrot” category would ever work. Very sad, but unfortunately true.
Nothing is set in stone or implemented yet. It was supposed to launch at VSS, but the lack of money caused the need for getting in new contract work to survive, which caused delays of many months. I hope we will have something within the next months.
Here is my rough idea for how it might go:
First we will implement it on macOS, not just for Apple Silicon. One reason is that continuing to support macOS beyond the year 2013 was so far the biggest money hole and worst personal and business decision one could have made. It is time we get something back. The primary reason is the large apparent interest in Apple Silicon support, as indicated by this discussion thread which has over 8900 views, by far more than any other discussion thread ever. If this is the feature that the largest group of people are waiting for, then it has the highest chance of people actually buying licenses quickly and thereby turning Psychtoolbox finances around. The fact that afaik all other software packages for macOS or Apple Silicon are in a way worse condition should hopefully help as an incentive. If this doesn’t work either then it is time to pull the plug on Psychtoolbox completely, because nothing will ever work.
I assume there will be kinks to be worked out with this at the beginning, given that I don’t have any experience whatsoever in implementing such a licensing scheme, and nobody I know does.
Once the kinks are worked out, we will apply the same scheme to Microsoft Windows.
And then we’ll see if that provides enough funding, or if we have to lock down some of the Linux variants as well. E.g., one future plan would be to eventually also provide a Psychtoolbox for Apple Silicon Macs under Linux, both to have a plan B / fallback in case Apple screws up macOS for Apple Silicon as badly as they did with Intel Macs, and also to provide a higher performance solution for people who want to make the most of their Apple Silicon. I don’t see a reason why that Linux variant should be free.
Wrt., what licenses specifically to offer, I’m up for feedback. E.g., I thought maybe something like a “lab license” for installation on ~ 5 machines for ~200 Euros/year, to cover needs of a small lab. I guess cheaper single individual licenses would also be useful. The question then immediately becomes what kind of support in case of trouble one could offer.
Whatever we do, we will be at least initially limited by the models that can actually be implemented by whatever 3rd party online shop or licensing manager we’d choose.
Such online shops legally act as “merchants of record” / resellers, just as our current Digistore24 online shop does. Iow. you don’t actually buy from us, but from that shop, we provide the service “on behalf of the shop”, the shop takes a cut of the profit, and the shop is responsible for handling international tax obligations and sales laws for hundreds of countries, credit card / payment transactions processing etc. Most such online shops are unfortunately not that well equipped to deal with all the special requirements that research labs often have, as opposed to Joe Average consumer on Amazon or similar.
Most flexible would be to operate all this ourselves, but I’m sure my employer would not be willing to take on that responsibility at this point, given how poor the Psychtoolbox business so far performed, and what kind of extra costs and risks to their business they would take. Sadly you can only throw money at problems if you have (earned) that money in the first place, so we are not in the luxurious position of being able to do that.
My intuition (not direct data) based on working in labs for the last 20 years or so is the following. Few labs run on macOS. You can get more bang for your buck elsewhere and far greater ability to upgrade components (which is pretty much impossible now on mac) and ability integrate a wider range of software and hardware. Most labs I am aware of run primarily on Windows (this comes down to many reasons I have mentioned before, so will not repeat here).
However, academia is a massive enclave of macOS users in terms of personal and office machines. These are very often used for developing code (by students through to Profs.).
I would suggest that licensing initially on both macOS and Windows would be by far the most sensible option. You would therefore target a very large % of your users in one go. If that does not work, then I agree with your view r.e. feasibility going on.
I do not think it sensible to start only with macOS as this is, in my experience, not used in labs (see above).
macOS and Windows are the ones to target for the $, in terms of lab based and office and personal machines. Labs will want the most up to date version etc.
Linux is being used more, but my impression is this is still the minority. Whilst I understand your “upgrade to Linux” philosophy, from my experience this is not really reflective of most of your users and the educational / university (and therefore research) sector. PTB also works really well compared to other systems on all operating systems.
Price… I need to give it some more thought, but I think you could ask for more for the “lab license”. Though it also needs to reflect the push to Python across the sector and therefore PsychoPy. If too expensive, folk will just accelerate the move to Python. PTB was pretty much the only game in town in terms of systems folk used for ages, but the landscape has changed with the massive increase in the use of Python. I say this even knowing the superiority of PTB. Just because something is better does not mean folk will use it. Python is the “go to” language these days.
I also think that a “site license” is needed also. Many software systems offer this. Furthermore, there needs to be a mechanism whereby students on courses can use PTB (this can be more then 5 or so). For example, I teach a course with ~30 each year. Also departments would by site licenses for use by all staff and students.
If you would like any help testing the macOS version I might be able to help, all be it in a small way. I use a macOS laptop and the lab is fully of various kit (but the lab runs on Windows).
I will ponder a bit more thoughts on pricing.
To reiterate, I think this is a good move for PTB. All be it I know, not how you would have liked things.
Just wondering if there is any update on the time-frame for Apple Silicon support and details on the licensing options and whether this will apply to just Apple or potentially Windows also. Obviously I understand we are just getting out of the summer and everyone is incredibly busy.
I would like to move forward with licensing of PTB for my lab. I also teach Matlab and Psychtoolbox and would like to learn how licensing at a university scale might work going forwards. I think we will stick with our current working installs for this academic year, but it would be good to start discussions about licensing going forwards.
Same with licensing options for Schools / Departments.
University administration moves at a glacial pace at the best of times!
I have a MacBook Pro with an M3 chip and cannot install PsychToolbox on it.
I get this error message:
Error using PsychtoolboxPostInstallRoutine (line 169)
Tried to setup on native Matlab or Octave for Apple Silicon 64-Bit ARM. This is
not supported.
Error in SetupPsychtoolbox (line 249)
PsychtoolboxPostInstallRoutine(1);
So this sounds like the M3 chip is the problem (new non-Intel generation of chips). However, a colleague of mine can run PsychToolbox on their M1 powered Macbook. Does this make the difference somehow?
You need to be using the Intel version of Matlab, and then you can continue to use the current (Intel-based) PTB. Shortly there will be a native version available for the new Apple Silicon computers, but it will require a paid license.
Yes, it does. However, you have to install the Intel version of Matlab. You can have both the ARM and Intel versions side by side.
The Intel one will install under Rosetta. From my experience this seems to work just fine. You will most likely take a speed hit in terms of raw processing, but I have not tested this.
PTB can then be installed with the intel version of Matlab. It seems to work fine for basic writing and testing of code. But I assume is unsupported and not recommended for running proper experiments. Note: I have not exhaustively tested it.
So currently I have both ARM and Intel versions of Matlab. The Intel on for use with PTB the ARM one for everything else.