OSX: what's the beef?

Hi folks,

Curious for more details on the following statement from the wiki
(http://psychtoolbox.org/SystemRequirements):

"...various issues in current OSX releases can not be reliably
resolved anymore. Use of OSX for demanding research wrt. visual timing
precision, stereoscopic multi-display presentation or other demanding
tasks is not recommended."

I've been using opengl via python on OSX, using highly-time-precise
light sensors to validate performance, and have as yet encountered no
issues. I'm genuinely curious to hear what issues others have
observed.

Mike

--
Mike Lawrence
Graduate Student
Department of Psychology & Neuroscience
Dalhousie University

~ Certainty is (possibly) folly ~
Thanks Mario, thats really useful info. 

Sounds like it can be extremely hit and miss as to getting things working on OSX. My rMBP appears to pass the compositor test in single screen mode. Will also test it on dual screen. Although the test code warns that even if it appears to pass the test it might not have. Its just used for development, as we run our VR on Windows. So, as long as it presents stimuli on the screen that I can see, and the timing is not to horrific it is good enough for me.

I was intrigued by the other posters (apparent) success with stereo on multiple monitors with OSX. Hopefully they will post a bit more information. 

With all the power of computers today it is a shame good timing is not more at the top of the list. Maybe with Occulus things might be prompted to change. 

But yes, for these companies it is the consumer market which is key. Thats where they make their money. Not through selling to vision scientists. 

P





---In PSYCHTOOLBOX@yahoogroups.com, <peterscarfe@...> wrote :

Thanks Mario, thats really useful info. 

Sounds like it can be extremely hit and miss as to getting things working on OSX. My rMBP appears to pass the compositor test in single screen mode. Will also test it on dual screen. Although the test code warns that even if it appears to pass the test it might not have. Its just used for development, as we run our VR on Windows. So, as long as it presents stimuli on the screen that I can see, and the timing is not to horrific it is good enough for me.

--

That specific new bug depends on os version and gpu, so far afaik OSX 10.9 + NVidia only, with no clear pattern on which NVidia gpu's screw up and which not. In general bugs are all over the place, e.g., the brokeness of AMD gpu's in mutli-display on 10.8 under some setups but not others etc.

In the end individual bugs don't matter to me, apart from the time wasted on trying to work around them and the increasingly high number of incidents on Windows and even more so on OSX where i just waste time and have to give up in the end after sinking days or weeks of time into it. What matters for long term sustainability is if the design or the operating system and the attitude and goals of the OS vendor align with our needs or are more or less in opposition, ie., if the system works with or against us, and if the vendor is at least marginally interested in working with us to improve his system for our purposes, or if the vendor couldn't care less.

For Microsoft Windows and Apple OSX the systems are not moving in a suitable direction and the vendors couldn't care less about our niche market.

For Linux the current system is much more well suited in most areas we care about, and there isn't a single vendor, but a large number of independent parties which collaborate on development of the system. Some couldn't care less about our applications. Some care to a large degree, because their own applications and goals align with ours and the success of their own projects or products depends on the same building blocks as ours. In general the development process is open to anybody which shows skills and willingness to contribute, so people like me can make a difference in how to system is evolving in areas we care about. Mid term and long term contributing to the os itself is the only way to have a good and somewhat sustainable basis for software like Psychtoolbox or other related neuro-science software.

My time is limited, so i will focus mostly on Linux in the future and not waste time on working around all those awful bugs in OSX or Windows. If OSX or Windows works well enough for somebodies specific purpose, they can continue to use it. If not, they know which system will get the most love and testing from me and where i'll try to give more attention in fixing issues if they should show up. On OSX or Windows i will provide the most basic level of support - time and resources permitting - to keep trivial stuff working. That said, by the end of September i will lose access to the last physical Windows test machine, so "testing" of Windows PTB will probably consist of compiling mex files in a virtual machine, typing "Screen" and see if it works or if it crashes with some "invalid mex file error". Whatever one can test in a VM under realistic conditions - which is not much. And my primary OSX development machine has the broken OSX 10.9 with its broken NVidia support, so i can't even do proper visual timing tests anymore with any confidence into the results. Can't wait for OSX 10.10 and the new horrors it will probably bring. Otoh the new lab i'm mostly hanging around at the moment has many Mac computers to explore the horror cabinet. I'm on a mission though to convince them to convert their experiment machines as quickly as possible to Linux for their and my mental sanity.

--

I was intrigued by the other posters (apparent) success with stereo on multiple monitors with OSX. Hopefully they will post a bit more information. 

--

Ja that would be good. Exact hardware and software setup, very detailed description of the testing procedure, scripts used etc. Most people i've seen testing timing with photo-diodes did it wrong one way or the other, which doesn't mean their results were neccessarily wrong, but that they would have (or have) missed certain types of timing bugs in their setups - their false negative rate would have been much higher than they might think. I've also seen a few published method papers which imho made nice contributions to the heap of unreproducible science.

On OSX in my experience the OS does nothing for you to provide you with synchronized video outputs on a dual display setup. The best you can hope for if everything is done right is that the scanout cycles stay in a fixed phase to each other, and that a sequence of random video mode switches (e.g., unplugging and replugging the displays, switching video resolution repeatedly etc.) may randomize the phase offset, so that at some point you may get a small enough error to be suitable - until the next system sleep/reboot/display off/on etc. With AMD graphics cards and the PsychtoolboxKernelDriver, PTB provides a home-grown method to synchronize the video outputs if you use that method - its not done automatically, you have to issues a Screen() command. On other gpu's it's all purely random. Even with synchronized displays, there's no os provided mechainsm to synchronize stimulus onset on both displays, so another collection of PTB hacks in the dual-display stereo modes tries to coerce the system into some halfway meaningful behaviour - not a 100% robust method, just as good as it gets on a fundamentally unsuitable system. To my knowledge no other vision science toolkit implements such tricks, but of course i don't know if some Python based VR toolkit might have implemented similar tricks. I do know from some Google hits that some other open source projects do use our kernel driver to work around issues on OSX, so as long as our kernel driver works, they will work. Of course Apple et al. are also working hard on making the kernel driver less useful :-( - On Intel gpu's e.g., the kernel driver is no longer used by default, as its use on modern Intel gpu's could literally damage the system it is used on to the point where a visit to the Apple repair shop might be needed to unbrick a machine. That's why there are no timing guarantees for OSX + Intel gpu's anymore with PTB, even on "bug free" OSX installations. Otoh some quick preliminary testing on a borrowed Apple MacBook Air 2013 last week showed that that machine works beautifully under Linux with apparently excellent timing and performance. It will need more testing to verify though.

--

With all the power of computers today it is a shame good timing is not more at the top of the list. Maybe with Occulus things might be prompted to change.

--

Maybe, maybe not. Occulus uses a single panel for their stereo rendering and that's probably a very good choice for the mental sanity of their development staff. The game company Valve now promotes "Steam OS" as the future gaming platform. Steam OS is based on Debian GNU/Linux for a good reason. Valve is working with gpu vendors to optimize their Linux products for Steam OS, so all this is certainly good for Linux. Many game and game engine companies lately do provide their game engines and increasingly games for Steam OS/Linux.

-mario

--
But yes, for these companies it is the consumer market which is key. Thats where they make their money. Not through selling to vision scientists. 

P


---In PSYCHTOOLBOX@yahoogroups.com, <peterscarfe@...> wrote :

I'd be happy to test code on Windows if they would help you in any way. I have access to a number of Windows machines in the lab. They have a variety of graphics cards / specs etc. So might provide a good test bed. All run Windows 7. 

---

Having somebody test stuff is of course never bad. The main problem is the system specific low-level code, mostly related to things like timing, precision (of color reproduction etc.) different display configurations, hardware interfacing (Kinect, i/o devices, etc.). For testing that you'd need the proper measurement equipment, e.g., Datapixx device, Kinects, other i/o hardware, cameras, and quite time consuming testing procedures. And if you find problems it's unlikely this could be remote debugged - at least it would be very painful for all involved parties.

--

My OSX system which appears to pass the compositor test is running 10.9 with an Nvidia graphics chip, so I guess I appear to be lucky with getting things to work well enough for testing. 

--

What NVidia card model is this? What doesn't work, e.g., is the GeForce GT-330M in early 2010 MacBookPro, but also much more recent cards failed. It would be nice to crowd-source such things on the new Wiki, so people don't have to guess.

--

We run our VR with single panel side-by-side stereo on Windows and it works great. I also use shutter goggles on another Windows 7 system with no issues. Shutter goggles tend to be a good test, as you notice any glitches immediately. 

--

Ja, single panel at least solves synchronization problems -> Nothing to sync with each other -> No stereo sync problems. On my MBP running frame-seq stereo just ends with completely messed up, massively tearing display -- obviously nothing Apple ever cared to test on their 2010 hardware.

--

Will be interested to see how Steam OS works. 

I'm getting more and more interested in using game engines to run experiments. Think I'm going to give Unity a shot once 5 comes out. Some colleagues in Engineering have said good things. 

But, as you say, it won't magically overcome OS limitations. 

--

No it won't overcome OS limitations at all. When it comes to low level stuff like timing and control it will certainly be worse than PTB. But for typical VR applications with no strong requirements for control or frame accurate timing it will be probably quite suitable. Some people here in Tue use it and are happy with it because i was told it is easy for beginners with no or minimal coding skills to setup VR experiments - well except for the license fees of course, as academic use is not free, only personal use. Also it's not free/open-source software, but that's a tradeoff everybody has to make for himself.

The good thing is that Unity is cross-platform afaik.

--

One can hope I guess. 

--

"Hope dies last" - old Russian saying.
-mario