Porting Psychtoolbox to Python

Bugs & Features
1

I broad question.

Having grown up on Matlab and Psychtoolbox I am (for better or worse) using Python more and more these days. The primary reasons are the vast push behind Python development in academia and beyond. Some of the core libraries I now use are only in Python (bar a clunky “call python from Matlab approach”).

“For better or worse” is that for many things I have encountered Python can be an utter headache. “Dependency hell” etc still exists. Why have one version of Python on your computer when you could in fact have 20?

One thing I always come back to Matlab for is Psychtoolbox. For me, PTB is still the utter gold standard for experiments.

I am aware of the existence of Octave, but this is obviously extraordinarily different from the Python ecosystem.

I was pondering if there were any plans, vague thoughts, dreams, hallucinations, about porting PTB to Python? It seems like this “could” be a good thing from the perspective of PTB in the current programming environment?

Obviously, “who is going to do this, and who will pay for it, who will maintain it”!?!?!? I am well aware of that.

The reason I raise this is that I have been porting another C++ library (haptic and visual) to Python using AI. It has been an effort and I have become even more aware of the limitations of AI. But, in the right hands and with the right checks, it can be a powerful tool.

Note: the library is much smaller than PTB. I am aware it is not a direct comparison. Just a triggering thought.

So, putting aside:

  1. “who is going to do this, and who will pay for it, who will maintain it”!?!?!?
  2. the various serious issues around AI
  3. etc.

Would pyPTB even be a sensible thought?

I am also aware of the “what will this get us when PTB works so well now” argument.

That might be a difficult question. I guess one thing would be user base and interoperability with the vast Python ecosystem. Like it or not, the user base is moving to Python. Possibly more monetary support (again I am aware of the problems historically with this, so am not selling this as a solution!).

Just some vague thoughts whilst I try and avoid some admin work.

Please note the many “if’s”, “buts” and “maybes”. This is very much crazy thinking and not in any way “whatever you might be thinking it is”. Just trying to avoid a crowd of people with burning torches kicking down my door tonight.

P

2

Hi Peter,

PTB already ships some of its mex modules to Python (PsychoPy uses PsychPortAudio and PsychHID), so it is feasible.

I tested doing this today with the PsychMetal project I’ve been working on. It is an experimental Screen-like toolkit that draws with Metal directly on Apple silicon (no OpenGL). It took a number of hours with heavy AI assistance, I split its native core into a host-neutral engine with two thin front ends, one MEX and one Python C API. The same engine now runs from MATLAB, Octave and Python, pip install psychmetal works (wheels for Python 3.10–3.14), and the same command inventory passes in all three hosts on my display.

All of your .m files would need to be rewritten, but AI can readily do this also. Validation and maintenance remain the real costs, as you say.

GitHub - keiths0/PsychMetal: Experimental direct Metal implementation of the Psychophysics Toolbox's Screen, on Apple silicon. No OpenGL. · GitHub if anyone wants to poke at it.

Keith

3

Screen is different from the other mex files, because of the extensive m-files wrapping its functionality, such as the monster PsychImaging function. Porting that, and maintaining parity across platforms was until now (in my eyes, i do not speak for Mario), the biggest bottleneck in porting PTB to Python. AI indeed takes away that hurdle in terms of code, but as you say, heavy validation and testing costs would remain. Perhaps, in terms of the wrappers being functionally correct, this could be caught by a special instrumented version of Screen that echoes its input arguments (and/or can serialize its config as far as different from default) so that automated tools can verify for a host of cases, across platforms, that identical demands lead to identical Screen invocations and configuration. Still, manual testing in the range of weeks (if i understand correctly) would be required on Mario’s hands to verify correct operation (different script interpreters may well interact badly with various drivers, etc).

Java is dead now anyway on newer matlabs too, so that is one less porting hassle.

4

Thanks Keith,

Yes, I did know that PsychPy was using various parts of PTB.

It is definitely a lot of work even for something much simpler than PTB.

P

5

This is kind of the approach I have taken. Because of AI’s propensity to lie and/or bend the facts and/or omit things, I have had it build in multiple checks and testbeds to validate things (which I can wee / check). Also, taking an adversarial approach has helped. One set of AI workers doing the work, the other trying to find holes. With some of the rendering we have even compared it to retraced scenes as part of the validation process and also the existing C++ library itself.

It has been hard work, but with all these checks I am getting much more confident. It really does however still need very strong guidance and checks as described, else you end up with something you cannot trust.

If you are running the C++ and Python version and they give identical results across a wide range of tests it seems like a reasonable test down to X decimal places.

Still feels rather uncomfortable though. It could be making subtle errors the millions of checks have not spotted. But I guess humans also make mistakes. Errors in the original C++ library have been found as part of the process. Some I already knew about prior to ever using AI, so it is reassuring it found those (and more).