I'm curious how well I can expect Psychtoobox to perform on a powerful Linux machine, but when the system is under load. This has recently come up in our lab as an upgrade to data acquisition system has led me to run both the stimulus and acquisition on the same machine. After some tests, it seems that PTB will drop at least a few frames when the system is under what I would consider a relatively light load. My question is, is this likely due to timing problems with our setup, or am I being overly optimistic in the performance I expect?
-> Likely timing problems with your setup, or the application you run causes trouble. That's about as specific as i can get. I have machines here, running ~7 years old medium class graphics cards under the open-source drivers with separate X-Screen's und KUbuntu 16.04-LTS which don't have trouble under additional graphics or other loads, or while starting up a firefox webbrowser. You should use the lowlatency kernel, as it can help a lot under certain loads for certain realtime tasks. But getting realtime behaviour or realtime under load is not something you get or don't get. It's highly specific to the workloads, the hardware you use, how that hardware is set up (also things like BIOS/EFI/Firmware settings, etc.). It only needs one underperforming component to destroy realtime behaviour.
E.g., none of the existing mainstream operating systems so far provides realtime scheduling for the graphics hardware, although at least the developers of the Intel graphics driver for Linux are currently working on some way to prioritize graphics tasks on some of the latest Intel graphics chips - work in progress... PTB's Priority command, especially on Linux and with the lowlatency kernel, can provide added realtime robustness for code that executes on the cpu. But if the reason for the dropped frames are actually related to additional graphics workloads, that alone won't help. E.g., you use the proprietary NVidia graphics drivers, so without access to the source code there isn't much one could do to find out if those have trouble with your data acquisition application, e.g., if that app uses some GUI with live-updating plots? In generally recommend AMD graphics cards or Intel graphics chips for lower performance needs, because they have good open-source drivers and that makes tweaking/diagnosing/fixing stuff much more easy.
The X-Server which is involved heavily in graphics operations is not realtime scheduled by default either, so switching it to RT scheduling would be worth a try, e.g., for priority 50:
sudo chrt --fifo --all-tasks --verbose -p 50 $(pidof Xorg)
Setting the gpu power management of the NVidia gpu in the nvidia-settings application to "maximum performance" could also help. This all under the assumption that it is something about your graphics activity of your data acquisition app, not something else.
So it all depends. Does your DAQ app do live plotting in its GUI or something gpu intense?
-mario
System details:
Kubuntu 16.04 LTS
Linux generic kernel 4.4.0-21 (I've tried lowlatency as well, basically the behavior)
NVIDIA GeForce GTX 970 (4GB)
NVIDIA drivers version 352.63
Intel quad-core i7
32GB main memory
2-display setup, completely independent X-screens.
Psychtoolbox Version 3.0.12
With no other applications running besides Matlab/PTB, things work great. There are no synchronization problems, and the standard deviation computed during PTB's initial testing are typically less than 0.01ms. Running long stimulus sets (on the order of 2 hours, updating every frame on the fly) results in no dropped frames whatsoever.
However, when running something simple, like `VBLSyncTest()`, things like opening a web browser cause dropped frames. Plots created by `VBLSyncTest()` indicate that Matlab is not scheduled for something like 4 or 5 frames when opening the browser. That is, the maximum missed deadline plotted can be up to 60ms.
Running our data acquisition application causes significantly more frequent frame drops, though their severity is not quite as bad (only 1 frame at a time). This is true even when increasing PTB's priority with the `Priority()` function, no matter how high I increase the requested priority. It is also true using the command-line `nice` or `renice` utilities.
Is this to be expected? Is it possible that further tweaks (e.g. `taskset`, changing CPU governors via `cpufreq-set`, or going to single-display) will be able to completely eliminate timing issues? Or is it unreasonable to expect that dropped frames will be eliminated entirely?
I'm curious how well I can expect Psychtoobox to perform on a powerful Linux machine, but when the system is under load. This has recently come up in our lab as an upgrade to data acquisition system has led me to run both the stimulus and acquisition on the same machine. After some tests, it seems that PTB will drop at least a few frames when the system is under what I would consider a relatively light load. My question is, is this likely due to timing problems with our setup, or am I being overly optimistic in the performance I expect?
-> Likely timing problems with your setup, or the application you run causes trouble. That's about as specific as i can get. I have machines here, running ~7 years old medium class graphics cards under the open-source drivers with separate X-Screen's und KUbuntu 16.04-LTS which don't have trouble under additional graphics or other loads, or while starting up a firefox webbrowser. You should use the lowlatency kernel, as it can help a lot under certain loads for certain realtime tasks. But getting realtime behaviour or realtime under load is not something you get or don't get. It's highly specific to the workloads, the hardware you use, how that hardware is set up (also things like BIOS/EFI/Firmware settings, etc.). It only needs one underperforming component to destroy realtime behaviour.
E.g., none of the existing mainstream operating systems so far provides realtime scheduling for the graphics hardware, although at least the developers of the Intel graphics driver for Linux are currently working on some way to prioritize graphics tasks on some of the latest Intel graphics chips - work in progress... PTB's Priority command, especially on Linux and with the lowlatency kernel, can provide added realtime robustness for code that executes on the cpu. But if the reason for the dropped frames are actually related to additional graphics workloads, that alone won't help. E.g., you use the proprietary NVidia graphics drivers, so without access to the source code there isn't much one could do to find out if those have trouble with your data acquisition application, e.g., if that app uses some GUI with live-updating plots?
The X-Server which is involved heavily in graphics operations is not realtime scheduled by default either, so switching it to RT scheduling would be worth a try:
System details:
Kubuntu 16.04 LTS
Linux generic kernel 4.4.0-21 (I've tried lowlatency as well, basically the behavior)
NVIDIA GeForce GTX 970 (4GB)
NVIDIA drivers version 352.63
Intel quad-core i7
32GB main memory
2-display setup, completely independent X-screens.
Psychtoolbox Version 3.0.12
With no other applications running besides Matlab/PTB, things work great. There are no synchronization problems, and the standard deviation computed during PTB's initial testing are typically less than 0.01ms. Running long stimulus sets (on the order of 2 hours, updating every frame on the fly) results in no dropped frames whatsoever.
However, when running something simple, like `VBLSyncTest()`, things like opening a web browser cause dropped frames. Plots created by `VBLSyncTest()` indicate that Matlab is not scheduled for something like 4 or 5 frames when opening the browser. That is, the maximum missed deadline plotted can be up to 60ms.
Running our data acquisition application causes significantly more frequent frame drops, though their severity is not quite as bad (only 1 frame at a time). This is true even when increasing PTB's priority with the `Priority()` function, no matter how high I increase the requested priority. It is also true using the command-line `nice` or `renice` utilities.
Is this to be expected? Is it possible that further tweaks (e.g. `taskset`, changing CPU governors via `cpufreq-set`, or going to single-display) will be able to completely eliminate timing issues? Or is it unreasonable to expect that dropped frames will be eliminated entirely?