Hi all,
I wonder if it is possible to easily build games in Psychtoolbox, using game engines like Unity, Unreal, etc. I found out that we could indeed use Unreal with Matlab through Simulink. However, I am not sure if it can be easily used in Psychtoolbox.
I think it would be great to have this feature so that we could easily run very engaging tasks through games, with minimal coding necessary.
Please let me know if you have any information or recommendations about this.
Thanks a lot!
Fatih
Hi Fatih,
This is currently not possible. However, Psychtoolbox does interface with the Horde3D game engine which allows high fidelity graphics to be displayed more easily.
However, this engine no longer seems to be under development and it does not get to your “no/minimal coding” criteria. I played around with Horde3DForPsychtoolbox a while back, but not for many years.
Additionally, game engines are not designed for high fidelity science research. For example, getting good timing etc. However, dependent on your use cases this might not be an issue - the fidelity available might be all you need. We run many of our VR experiments using Unreal Engine and for our purposes for these experiments, this is fine. For other VR experiments we use the great functionality in PTB.
P
Horde3D is still under active development and maintenance. It’s just that nobody seems to have updated the actual download links on their website since years. They announce v1.0.0 as the latest, whereas their GitHub page has a v2.0.0 release since last year, and git branches show development as recent as february this year.
That said, my Psychtoolobox interface hasn’t been updated since years, so is lacking all advances since v1.0.0-beta5. But the interface works, and it has been used by at least two labs in Tuebingen over the years for spatial navigation and VR experiments. I use it to make my VR tests a bit less dull…
In general, as Peter said, the main problem of regular stand-alone game engines is the lack of low-level control and timing control etc., not at the level that Psychtoolbox provides. To get some ease of use and functionality for complex 3D scene rendering, one needs to tightly integrate the 3D engine into Psychtoolbox. Specifically a scripting interface to “remote control” the engine from Octave/Matlab, and the ability to direct rendering of the final mono or stereo images into Psychtoolbox virtual framebuffers / OpenGL FBO’s. This way PTB’s imaging pipeline can do any needed post-processing of the rendered images for a given experimental setup, and stay in control of displays, timing etc.
Most standard engines like Unity or Unreal seem to be ill-suited for this, they don’t play well with 3rd party software and are not meant to integrate into it. At least that’s what a quick google search wrt. Unity or Unreal seems to say.
Horde3D, although less advanced than those, is meant to integrate well into 3rd party apps like Psychtoolbox and to be easily scriptable. That’s why it was chosen as afaik so far the only 3D engine.
Hope this answers the question.