Multi-threaded PTB

1
Hi all,

Sorry if this is a bit obvious, but I've been looking around the internets for a while now without finding a solution.

Is it possible to run PTB functions in multiple-threads? Is it even possible to explicitly run functions in parallel in Matlab, and can they communicate?

I'm having issues getting triggering peripherals (EEG, TMS) at the exact instant that a video frame is displayed, and I feel like having separate concurrent threads for triggering, display, and input logging would be the optimal solution.

Unfortunately I am having difficulty figuring out whether this is actually possible, does any one have any experience with this?

Cheers, Matt
2
> Is it possible to run PTB functions in multiple-threads? Is it even possible to explicitly run functions in parallel in Matlab, and can they communicate?

matlab doesn't provide a concurrency abstraction. your problem doesn't need it, but you could use the www.mathworks.com/products/parallel-computing/ toolbox or one of its FOSS equivalents to run communicating matlab processes on separate machines (or maybe cores on one machine). i doubt whether there's any network protocol you could hack to get reliable millisecond-level communication among them.

you can write a mex function and use c threads, but the matlab runtime isn't threadsafe (afaik), so your spawned threads have to be careful not to make any mx calls. i believe ptb spawns off some worker threads...

your problem is easy though, all you want is fast access to some ttl hardware (http://psychtoolbox.org/faqttltrigger)

> I'm having issues getting triggering peripherals (EEG, TMS) at the exact instant that a video frame is displayed,

there is no such thing as that exact instant, the frame is progressively disclosed over nearly the whole duration of that frame.

> and I feel like having separate concurrent threads for triggering, display, and input logging would be the optimal solution.

separate threads would be conceptually much cleaner, i agree, but definitely not necessary. what problem are you having?

> Unfortunately I am having difficulty figuring out whether this is actually possible, does any one have any experience with this?

lots of us. just set a ttl line hi on one side of your flip, and lo on the other. this will also give you a measurement of computation time headroom.

-e
3
Thanks for your reply Mario, could you please point me in the direction of the documentation for background thread input logging?

I really don't want to do anything out of the ordinary, and I'm sure that the current work-arounds would suffice, in fact I'll probably end up doing it serially. That being said, I don't think that the existence of work-arounds should stifle conversation about perhaps more ideal solutions.

I've had a bit of a look at the Parallel Computing Toolbox, and you're right, a lot of it is not particularly useful. The most promising technique that I can find for use with PTB are spmd statements. Here's a simple example of a function that does three operations concurrently: displaying a timestamp, an arbitrary complex operation (fft), and streaming of KbCheck data into a csv file. It needs 3 or more cores to run.

function ptrial
    matlabpool(3);
    disp('Started');
    baseTime=GetSecs;
    spmd
        labBarrier();
        switch labindex
            case 1
                for i=1:10
                    while GetSecs-baseTime<i
                    end
                    disp(GetSecs-baseTime);
                end
            case 2
                outputFile = fopen('kbcheck.csv','wt');
                while GetSecs-baseTime<10
                    [keyIsDown,secs,keyCode,deltaSecs] = KbCheck();
                    if keyIsDown
                        keys = KbName(keyCode);
                        secs = secs-baseTime;
                        if ischar(keys)
                            fprintf(outputFile,'%s,%f,%f',keys,secs,deltaSecs);
                        else
                            for j=1:size(keys,2)
                                fprintf(outputFile,'%s,%f,%f,',keys{j},secs,deltaSecs);
                            end
                        end
                        fprintf(outputFile,'\n');
                    end
                end
                disp(['File closed: ' num2str(fclose(outputFile))]);
            case 3
                while GetSecs-baseTime<10
                    fft(rand(20000000,1));
                    disp(['FFT loop completed @ ' num2str(GetSecs-baseTime)]);
                end
        end
    end
    disp('Done');
    matlabpool close;
end

Unfortunately, from my cursory investigation, it seems that Screen cannot be called from inside such a statement, due to restrictions on 'transparency', a term which I'm still not entirely clear on. The function will get as far as opening a window, but will not start displaying anything before crashing. I assume that this is because Screen is a mex operation, whereas KbCheck is not.

So yeah, unless someone can shed some light on potential solutions to this, it seems that the buck stops there in terms of spmd.

Additionally, I can see parfor loops having some utility in video preloading, although I haven't given it a go myself.

--- In psychtoolbox@yahoogroups.com, "Mario" <mario.kleiner@...> wrote:
>
>
>
> Afaik there isn't any explicit multi-threading support in Matlab, apart from parallel computing toolbox stuff, which is not that useful or elegant for what you want. Psychtoolbox itself uses multi-threading internally in various places to simplify things or make use of multi-core machines.
>
> Input logging can be done in the background, depending on your input device, e.g., keyboards, keypads, mouse or joystick buttons, various reponse boxes, stuff that comes in over the (usb-)serial port. Ptb can collect and timestamp such data on a background thread and you can retrieve it at convenient times.
>
> As far as display goes, you can schedule stimulus display in the background via Screen('AsyncFlipBegin), executed on a parallel background thread, but parallelism between display and trigger emission won't help you if your trigger is supposed to be locked to stimulus onset, because your hypothetical trigger thread would have to wait for a signal from the display thread that stimulus onset has happened, so you'd just have a more complicated and error prone way of waiting for a Screen('Flip') to return.
>
> Essentially you have two options:
>
> 1. Classic: Screen('Flip') then immediately after that, emit your trigger. This is reliable but if your output hardware has long latencies, or your operating system/machine has bad execution timing, the trigger would be somewhat delayed.
>
> 2. Optimistic about the future: Use ...
> Screen('AsyncFlipBegin', window, when); to schedule stimulus onset at target time t >= when.
> Then use tTrigger = PredictVisualOnsetForTime(window, when); to get an estimated time 'tTrigger' when stimulus onset will happen,
> then use WaitSecs('UntilTime', tTrigger - slack); to wait until some 'slack' seconds before the wanted trigger time, with a slack properly chosen for your sluggish hardware, if the hardware is sluggish,
> then emit your trigger and hope for the best. tVisual = Screen('AsyncFlipEnd', window); would wait for stimulus onset and provide you with the true stimulus onset time 'tVisual' and you could hope that tVisual and the time of trigger emission now matches.
>
> Obviously if 'Flip' misses the presentation deadline by a frame, your optimistically emitted trigger would now be one frame too early.
>
> I think 'Classic' is what almost anybody uses, because with proper hardware and system setup it is good enough and much simpler.
>
> I intend to add some more clever trigger emission support to ptb, probably sometime before the end of this year (no guarantees), but multi-threading is just one possible part of the solution for that, and depending on operating system and hardware probably not much more reliable, just hopefully a bit more easy to use. The idea would be to have a unified api to define what one wants and the actual implementation can use single-threading, multi-threading, special operating system facilities, or special capabilities of research hardware.
>
> Of course you can also throw special hardware at the problem if you have the money or skills.
>
> But most likely your problem is not where you think it is, as Diederick and others suggested, so maybe you should describe your system setup, code and observed results in more detail for others to help you.
>
> -mario
>
>
> --- In psychtoolbox@yahoogroups.com, "pmcgoots" <pmcgoots@> wrote:
> >
> > Hi all,
> >
> > Sorry if this is a bit obvious, but I've been looking around the internets for a while now without finding a solution.
> >
> > Is it possible to run PTB functions in multiple-threads? Is it even possible to explicitly run functions in parallel in Matlab, and can they communicate?
> >
> > I'm having issues getting triggering peripherals (EEG, TMS) at the exact instant that a video frame is displayed, and I feel like having separate concurrent threads for triggering, display, and input logging would be the optimal solution.
> >
> > Unfortunately I am having difficulty figuring out whether this is actually possible, does any one have any experience with this?
> >
> > Cheers, Matt
> >
>

4
> Thanks for your reply Mario, could you please point me in the direction
> of the documentation for background thread input logging?

it's happening automatically -- that's where GetChar, etc are storing keypresses until you ask for them. it uses java's concurrency, i guess another option for you if that's more your thing than c.

> I don't think that the existence
> of work-arounds should stifle conversation about perhaps more ideal
> solutions.

emitting a ttl timed exactly with the flips is such a coupled task that using concurrency just for that is way overcomplicating. concurrency is extremely hairy to get right and only now are the most modern languages (haskell) discovering abstractions that improve code safety and simplicity/readability in a concurrent context. if the workers don't need to interact, the problems go away, but as soon as you have to coordinate them, you have to be extremely careful about deadlock, not overwriting information in one task during the time that some other task is depending on that information not changing, etc.

as mario said, in your case, the flipping process has to tell the trigger process each time it flips, so you haven't saved anything from just triggering in the flipping process!

the times i wish i had concurrency are things like opening a reward valve for a timed duration while still showing stimuli, etc. that's a case where two different kinds of state gets all unpleasantly jumbled in the drawing loop. checking for responses and their correctness is another example, or playing a sound just when a beambreak is blocked, or keeping a log of mouse movements, etc.

> spmd

http://en.wikipedia.org/wiki/Referential_transparency_(computer_science)

as you discovered, they put all sorts of limitations on spmd code, i believe in an effort to prevent you from running into the types of problems described above. the "transparency" they describe looks like a typical mathworks-mangling of something that would have been useful if done correctly, in this case "referential transparency," which means "calling this function doesn't have any side-effects, all it does is compute a value, and given the same arguments, that value is guaranteed to be the same. so it is guaranteed not to do things like look up web pages, read from a random number generator, etc." those are all side effects that would make a visible change out in the real world.

just an aside, in haskell, code is pure (referentially transparent) by default. you sequester side effecting code, so it can't infect everything -- this is enforced by the compiler! pure code can be automatically parallelized! often people make the distinction between 'parallel' and 'concurrent' -- parallel being just those cases where the workers are pure, so you don't have the usual thorny coordination problems, a compiler can safely apply very dramatic optimizations, etc.

side effects, though, can be used by workers to communicate arbitrarily, opening up all the problems.

back to spmd, it looks like they were only halfhearted about trying to prevent side effects -- they say you can't 'save', but it looks like you found fopen/fprintf worked.

http://www.mathworks.com/help/distcomp/programming-tips_brukbnp-9.html#brukbnp-12

i don't see them saying you can't call java objects or mex functions, but if they let you call them, there's a huge hole letting you have arbitrary side effects. if you opened a ptb window, you got a mex call working, so if you insisted, you could use this to work around their attempts to stop you. same for writing to a file, writing to the network (or screen), etc.

but you are really forcing a square peg into a round hole here. note spmd = "single program multiple data" -- it is meant to work on very regular problems, like applying the same transformation at every point in an array or something -- exactly like the vectorized simd functions we all know and love -- fft, sum, etc, that run on a chip designed to do the same thing to several datapoints all at once, not as a loop on the cpu working one at a time. note how your program was really three different programs.

the toolbox gives you 'batch' for this:
http://www.mathworks.com/help/distcomp/batch.html

and i don't see that it limits what you can do in each worker, so it looks like you can go nuts running into nondeterministic concurrency bugs. :)

> Additionally, I can see parfor loops having some utility in video
> preloading, although I haven't given it a go myself.

not sure what you mean by preloading (computing a complicated stimulus offline? sure. but reading a huge stimulus from disk? won't help.), but parfor requires the workers to be both pure and independent, so it wouldn't work for the concurrency we've been discussing.

http://www.mathworks.com/help/distcomp/getting-started-with-parfor.html#brdqg4q-1

-e
5
Mostly what Erik said.

With one correction: GetChar is not useful for background input collection if you want it timestamped. It does provide keyboard character input to you, read from the applications event queue, via some Java code on some platforms and some other hacks on others, and that queue is filled asynchronously by the operating system if all goes well. But timestamps may be wildly imprecise, special keys other than typical characters are not recognized, and input will get lost on some systems if the Matlab application window loses keyboard input focus - or not. There are so many different ways and workarounds to make GetChar work on different setups that i can't remember which one happens when on which setup - i each time have to read the code i wrote to untangle it for a given setup. We went through great pain in the past to keep it working in a backwards compatible way, but related help texts tells you to only use it for simple things, maybe asking your subject for its name or age or some simple non-time critical response etc., but even there our other functions might be more appropriate in new code. There is not much good to be said about it at all, it really mostly only exists for backwards compatibility for old code.

For reliable timestamped keyboard, keypad, mouse button, (often joystick) button input (== regular "button" stuff) we have keyboard queues, with KbEventGet/KbEventFlush/KbEventAvail as rather convenient interfaces in GetChar's spirit (queueing all input, reading it from the queue at convenient times) and KbQueueCheck, KbQeueWait, KbQueueFlush etc. for a more KbCheck like interface. All demonstrated in KbQueueDemo. There are still limitations here: On OSX you can have only one such input device handled like that, on MS-Windows you cannot distinguish different keyboards or different mice. On Linux there are no known limitations on the number of addressable devices and you can also treat different keyboards, mice etc. individually.

For (i think) most common response boxes we have dedicated drivers with similar queuing/query functionality as with keyboard queues: CMUBox for most stuff that can connect via serial port or a simulated "serial over USB" port. We also have examples for collecting arbitrary timestamped data or certain trigger input from serial ports, e.g., serial port connected eyetrackers, there are demos for that in the PsychDemos folder with self-explanatory names.

All this works by having internal threads running in our IOPort or PsychHID mex files and timestamping/logging data.

For some response boxes we have extra drivers which make use of those boxes logging and internal timestamping capabilities: PsychRTBox() for the RtBox, and CedrusResponseBox for Cedrus devices. However i don't recommend Cedrus devices at all, in my and others experiences they are pretty buggy and problematic in most functionality that would make them interesting in the first place. The CMUBox driver can address Cedrus boxes in a much more dumb way, which throws away almost all of their sophisticated and utterly unreliable functionality and treats them as really dumb and cheap response boxes.

As i said before, the planned improvements to the IOPort driver should allow more async stuff on the input/output side, once i get around working on it. Erik, your input/wishlists are welcome. Maybe we should have some kind of scrap page on the Wiki where you and others could collect use cases/wishes, or maybe as an issue in the issue tracker.

With respect to the multi-threading stuff, the difficult part is the communication and synchronization between threads, bug inducing and brain twisting enough for most programmers, but additionally you have to take realtime aspects into account, e.g., proper priorization of threads, so they don't steal resources and execution time from each other at the wrong moments. That's not so important for regular parallel apps, but for timing-sensitive apps it adds extra headache. Especially on MS-Windows where there aren't many useable priority levels for our purpose (about 2 to 3, depending if you run as Admin or not), there's a lot of fun to be had. And then there are all the clock issues on multi-core execution on Windows and on and on.

I could see some value of a very well done explicit threading implementation in Matlab/Octave when used by users with lots of experience in multi-threaded realtime programming, or for very basic no brainer tasks. But then most users are not threading experts, so i would assume to most people it would provide a much more effective way to torture oneselve.

As far as i understood, the parallel computing toolbox doesn't use threads as building blocks, but instances of full Matlab processes, with separate address spaces and other resources and very limited communication channels. That's rather heavy weight, probably not very realtime friendly on some os'es and probably bound to cause all kinds of funny interactions with hardware, e.g., graphics hardware. I think parallel computing toolbox is really only meant for parallel computations with very limited and well-defined i/o. Just to give another example wrt. GetSecs and clocks on Windows: The mechanisms we need to use to detect and compensate for broken clock hardware on Windows do need to communicate their state across different mex files. Doing this across different worker processes instead of threads is impossible, so your GetSecs and other timestamps collected on different workers may refer to different clocks with different reference points -> timestamps are no longer comparable anymore.

-mario


--- In psychtoolbox@yahoogroups.com, "Erik" <e_flister@...> wrote:
>
> > Thanks for your reply Mario, could you please point me in the direction
> > of the documentation for background thread input logging?
>
> it's happening automatically -- that's where GetChar, etc are storing keypresses until you ask for them. it uses java's concurrency, i guess another option for you if that's more your thing than c.
>
> > I don't think that the existence
> > of work-arounds should stifle conversation about perhaps more ideal
> > solutions.
>
> emitting a ttl timed exactly with the flips is such a coupled task that using concurrency just for that is way overcomplicating. concurrency is extremely hairy to get right and only now are the most modern languages (haskell) discovering abstractions that improve code safety and simplicity/readability in a concurrent context. if the workers don't need to interact, the problems go away, but as soon as you have to coordinate them, you have to be extremely careful about deadlock, not overwriting information in one task during the time that some other task is depending on that information not changing, etc.
>
> as mario said, in your case, the flipping process has to tell the trigger process each time it flips, so you haven't saved anything from just triggering in the flipping process!
>
> the times i wish i had concurrency are things like opening a reward valve for a timed duration while still showing stimuli, etc. that's a case where two different kinds of state gets all unpleasantly jumbled in the drawing loop. checking for responses and their correctness is another example, or playing a sound just when a beambreak is blocked, or keeping a log of mouse movements, etc.
>
> > spmd
>
> http://en.wikipedia.org/wiki/Referential_transparency_(computer_science)
>
> as you discovered, they put all sorts of limitations on spmd code, i believe in an effort to prevent you from running into the types of problems described above. the "transparency" they describe looks like a typical mathworks-mangling of something that would have been useful if done correctly, in this case "referential transparency," which means "calling this function doesn't have any side-effects, all it does is compute a value, and given the same arguments, that value is guaranteed to be the same. so it is guaranteed not to do things like look up web pages, read from a random number generator, etc." those are all side effects that would make a visible change out in the real world.
>
> just an aside, in haskell, code is pure (referentially transparent) by default. you sequester side effecting code, so it can't infect everything -- this is enforced by the compiler! pure code can be automatically parallelized! often people make the distinction between 'parallel' and 'concurrent' -- parallel being just those cases where the workers are pure, so you don't have the usual thorny coordination problems, a compiler can safely apply very dramatic optimizations, etc.
>
> side effects, though, can be used by workers to communicate arbitrarily, opening up all the problems.
>
> back to spmd, it looks like they were only halfhearted about trying to prevent side effects -- they say you can't 'save', but it looks like you found fopen/fprintf worked.
>
> http://www.mathworks.com/help/distcomp/programming-tips_brukbnp-9.html#brukbnp-12
>
> i don't see them saying you can't call java objects or mex functions, but if they let you call them, there's a huge hole letting you have arbitrary side effects. if you opened a ptb window, you got a mex call working, so if you insisted, you could use this to work around their attempts to stop you. same for writing to a file, writing to the network (or screen), etc.
>
> but you are really forcing a square peg into a round hole here. note spmd = "single program multiple data" -- it is meant to work on very regular problems, like applying the same transformation at every point in an array or something -- exactly like the vectorized simd functions we all know and love -- fft, sum, etc, that run on a chip designed to do the same thing to several datapoints all at once, not as a loop on the cpu working one at a time. note how your program was really three different programs.
>
> the toolbox gives you 'batch' for this:
> http://www.mathworks.com/help/distcomp/batch.html
>
> and i don't see that it limits what you can do in each worker, so it looks like you can go nuts running into nondeterministic concurrency bugs. :)
>
> > Additionally, I can see parfor loops having some utility in video
> > preloading, although I haven't given it a go myself.
>
> not sure what you mean by preloading (computing a complicated stimulus offline? sure. but reading a huge stimulus from disk? won't help.), but parfor requires the workers to be both pure and independent, so it wouldn't work for the concurrency we've been discussing.
>
> http://www.mathworks.com/help/distcomp/getting-started-with-parfor.html#brdqg4q-1
>
> -e
>
6
Hi Mario, thanks for your all your help. I don't really have a problem, per se. I'm not going into this blind, I have done a software engineering degree. I'm in the very early stages, and am really just wanting to try and understand how things are built before I dive in. Thanks, this has really helped!

--- In psychtoolbox@yahoogroups.com, "Mario" <mario.kleiner@...> wrote:
>
>
>
> --- In psychtoolbox@yahoogroups.com, "pmcgoots" <pmcgoots@> wrote:
> >
> > Thanks Erik and Mario, it does sound like a bit more trouble than it's
> > worth, maybe still needs a bit of maturing.
> >
>
> I so far get the feeling that you are mostly missing our points.
>
> Wasn't your original problem that you can't manage to emit exactly timed triggers at visual stimulus onset with some unspecified i/o hardware on some unspecified computer/operating system/graphics card/display setup, according to some unspecified way you measured this?
>
> And you proposed that putting response collection, stimulus presentation and triggering into different threads would solve your problem. And we told you that you can already do response collection asynchronously in many cases, so that problem would probably be solved. And we told you that there would be *zero* benefit in separating stimulus presentation and simple trigger emission into separate threads, even with the most perfect threading framework one could think of, it would be always more complicated but in no way more reliable.
>
> Also we tried unsuccessfully to get you to specify your actual triggering problem, setup, measurement approach etc. more precisely, as we think your problem is not where you think your problem is.
>
> > Regarding the GetSecs problem, I feared that might be the case. Do you
> > know if the same issue exists with the inbuilt tic/toc?
>
> I don't see how you could use tic and toc as a replacement for GetSecs? When i mean GetSecs i mean the underlying clock and timing code in WaitSecs, Screen, PsychPortAudio, etc. as well. Or how you'd synchronize time across workers with a tic/toc approach?
>
> But on reliable, perfectly working clock hardware, GetSecs and tic / toc on a recent Matlab version nowadays might be of comparable accuracy, in the past, tic/toc were less accurate. On slightly broken clock hardware on MS-Windows, GetSecs et al. can detect and compensate to some degree for flaws if used as intended, ie., not in some weird multi-process configuration via parallel computing toolbox. In such a setup, GetSecs, tic and toc will be just equally broken if running on flawed hardware.
>
> Linux and OSX don't suffer the effects of such clock flaws as far as my experience goes. Linux has sophisticated compensation mechanisms at the operating system level, and OSX only runs on Apple hardware which so far seems to be free of design flaws in that area.
>
> >
> > Regardless, the KbQueue looks like it will do anything that we could do
> > with the rather simple parallelism in Matlab.
> >
>
> I don't know what's missing for you, but so far you didn't specify your task precisely in the first place.
>
> > For concurrently performing tasks over frame, do you think there's any
> > utility to using the job framework?
> >
>
> No. But i would recommend you reading the introductory material on parallel computing toolbox, to understand why none of this make any sense at least for the problems i believe you try to solve, which brings us back to the first paragraph of my response.
>
> -mario
>
> > e.g.
> > job = createJob();
> > task1 = createTask(gob,@,1);
> > waitForState(task1,'running');
> > %Do other things
> >
> > function longerTask(input)
> > %Multi-frame task here.
> > end
> >
> > Although I'm not exactly sure how long this task creation will take and
> > such.
> >
> > --- In psychtoolbox@yahoogroups.com, "Mario" wrote:
> > >
> > >
> > >
> > > Mostly what Erik said.
> > >
> > > With one correction: GetChar is not useful for background input
> > collection if you want it timestamped. It does provide keyboard
> > character input to you, read from the applications event queue, via some
> > Java code on some platforms and some other hacks on others, and that
> > queue is filled asynchronously by the operating system if all goes well.
> > But timestamps may be wildly imprecise, special keys other than typical
> > characters are not recognized, and input will get lost on some systems
> > if the Matlab application window loses keyboard input focus - or not.
> > There are so many different ways and workarounds to make GetChar work on
> > different setups that i can't remember which one happens when on which
> > setup - i each time have to read the code i wrote to untangle it for a
> > given setup. We went through great pain in the past to keep it working
> > in a backwards compatible way, but related help texts tells you to only
> > use it for simple things, maybe asking your subject for its name or age
> > or some simple non-time critical response etc., but even there our other
> > functions might be more appropriate in new code. There is not much good
> > to be said about it at all, it really mostly only exists for backwards
> > compatibility for old code.
> > >
> > > For reliable timestamped keyboard, keypad, mouse button, (often
> > joystick) button input (== regular "button" stuff) we have keyboard
> > queues, with KbEventGet/KbEventFlush/KbEventAvail as rather convenient
> > interfaces in GetChar's spirit (queueing all input, reading it from the
> > queue at convenient times) and KbQueueCheck, KbQeueWait, KbQueueFlush
> > etc. for a more KbCheck like interface. All demonstrated in KbQueueDemo.
> > There are still limitations here: On OSX you can have only one such
> > input device handled like that, on MS-Windows you cannot distinguish
> > different keyboards or different mice. On Linux there are no known
> > limitations on the number of addressable devices and you can also treat
> > different keyboards, mice etc. individually.
> > >
> > > For (i think) most common response boxes we have dedicated drivers
> > with similar queuing/query functionality as with keyboard queues: CMUBox
> > for most stuff that can connect via serial port or a simulated "serial
> > over USB" port. We also have examples for collecting arbitrary
> > timestamped data or certain trigger input from serial ports, e.g.,
> > serial port connected eyetrackers, there are demos for that in the
> > PsychDemos folder with self-explanatory names.
> > >
> > > All this works by having internal threads running in our IOPort or
> > PsychHID mex files and timestamping/logging data.
> > >
> > > For some response boxes we have extra drivers which make use of those
> > boxes logging and internal timestamping capabilities: PsychRTBox() for
> > the RtBox, and CedrusResponseBox for Cedrus devices. However i don't
> > recommend Cedrus devices at all, in my and others experiences they are
> > pretty buggy and problematic in most functionality that would make them
> > interesting in the first place. The CMUBox driver can address Cedrus
> > boxes in a much more dumb way, which throws away almost all of their
> > sophisticated and utterly unreliable functionality and treats them as
> > really dumb and cheap response boxes.
> > >
> > > As i said before, the planned improvements to the IOPort driver should
> > allow more async stuff on the input/output side, once i get around
> > working on it. Erik, your input/wishlists are welcome. Maybe we should
> > have some kind of scrap page on the Wiki where you and others could
> > collect use cases/wishes, or maybe as an issue in the issue tracker.
> > >
> > > With respect to the multi-threading stuff, the difficult part is the
> > communication and synchronization between threads, bug inducing and
> > brain twisting enough for most programmers, but additionally you have to
> > take realtime aspects into account, e.g., proper priorization of
> > threads, so they don't steal resources and execution time from each
> > other at the wrong moments. That's not so important for regular parallel
> > apps, but for timing-sensitive apps it adds extra headache. Especially
> > on MS-Windows where there aren't many useable priority levels for our
> > purpose (about 2 to 3, depending if you run as Admin or not), there's a
> > lot of fun to be had. And then there are all the clock issues on
> > multi-core execution on Windows and on and on.
> > >
> > > I could see some value of a very well done explicit threading
> > implementation in Matlab/Octave when used by users with lots of
> > experience in multi-threaded realtime programming, or for very basic no
> > brainer tasks. But then most users are not threading experts, so i would
> > assume to most people it would provide a much more effective way to
> > torture oneselve.
> > >
> > > As far as i understood, the parallel computing toolbox doesn't use
> > threads as building blocks, but instances of full Matlab processes, with
> > separate address spaces and other resources and very limited
> > communication channels. That's rather heavy weight, probably not very
> > realtime friendly on some os'es and probably bound to cause all kinds of
> > funny interactions with hardware, e.g., graphics hardware. I think
> > parallel computing toolbox is really only meant for parallel
> > computations with very limited and well-defined i/o. Just to give
> > another example wrt. GetSecs and clocks on Windows: The mechanisms we
> > need to use to detect and compensate for broken clock hardware on
> > Windows do need to communicate their state across different mex files.
> > Doing this across different worker processes instead of threads is
> > impossible, so your GetSecs and other timestamps collected on different
> > workers may refer to different clocks with different reference points ->
> > timestamps are no longer comparable anymore.
> > >
> > > -mario
> > >
> > >
> > > --- In psychtoolbox@yahoogroups.com, "Erik" e_flister@ wrote:
> > > >
> > > > > Thanks for your reply Mario, could you please point me in the
> > direction
> > > > > of the documentation for background thread input logging?
> > > >
> > > > it's happening automatically -- that's where GetChar, etc are
> > storing keypresses until you ask for them. it uses java's concurrency,
> > i guess another option for you if that's more your thing than c.
> > > >
> > > > > I don't think that the existence
> > > > > of work-arounds should stifle conversation about perhaps more
> > ideal
> > > > > solutions.
> > > >
> > > > emitting a ttl timed exactly with the flips is such a coupled task
> > that using concurrency just for that is way overcomplicating.
> > concurrency is extremely hairy to get right and only now are the most
> > modern languages (haskell) discovering abstractions that improve code
> > safety and simplicity/readability in a concurrent context. if the
> > workers don't need to interact, the problems go away, but as soon as you
> > have to coordinate them, you have to be extremely careful about
> > deadlock, not overwriting information in one task during the time that
> > some other task is depending on that information not changing, etc.
> > > >
> > > > as mario said, in your case, the flipping process has to tell the
> > trigger process each time it flips, so you haven't saved anything from
> > just triggering in the flipping process!
> > > >
> > > > the times i wish i had concurrency are things like opening a reward
> > valve for a timed duration while still showing stimuli, etc. that's a
> > case where two different kinds of state gets all unpleasantly jumbled in
> > the drawing loop. checking for responses and their correctness is
> > another example, or playing a sound just when a beambreak is blocked, or
> > keeping a log of mouse movements, etc.
> > > >
> > > > > spmd
> > > >
> > > >
> > http://en.wikipedia.org/wiki/Referential_transparency_(computer_science)
> > > >
> > > > as you discovered, they put all sorts of limitations on spmd code, i
> > believe in an effort to prevent you from running into the types of
> > problems described above. the "transparency" they describe looks like a
> > typical mathworks-mangling of something that would have been useful if
> > done correctly, in this case "referential transparency," which means
> > "calling this function doesn't have any side-effects, all it does is
> > compute a value, and given the same arguments, that value is guaranteed
> > to be the same. so it is guaranteed not to do things like look up web
> > pages, read from a random number generator, etc." those are all side
> > effects that would make a visible change out in the real world.
> > > >
> > > > just an aside, in haskell, code is pure (referentially transparent)
> > by default. you sequester side effecting code, so it can't infect
> > everything -- this is enforced by the compiler! pure code can be
> > automatically parallelized! often people make the distinction between
> > 'parallel' and 'concurrent' -- parallel being just those cases where the
> > workers are pure, so you don't have the usual thorny coordination
> > problems, a compiler can safely apply very dramatic optimizations, etc.
> > > >
> > > > side effects, though, can be used by workers to communicate
> > arbitrarily, opening up all the problems.
> > > >
> > > > back to spmd, it looks like they were only halfhearted about trying
> > to prevent side effects -- they say you can't 'save', but it looks like
> > you found fopen/fprintf worked.
> > > >
> > > >
> > http://www.mathworks.com/help/distcomp/programming-tips_brukbnp-9.html#b\
> > rukbnp-12
> > > >
> > > > i don't see them saying you can't call java objects or mex
> > functions, but if they let you call them, there's a huge hole letting
> > you have arbitrary side effects. if you opened a ptb window, you got a
> > mex call working, so if you insisted, you could use this to work around
> > their attempts to stop you. same for writing to a file, writing to the
> > network (or screen), etc.
> > > >
> > > > but you are really forcing a square peg into a round hole here.
> > note spmd = "single program multiple data" -- it is meant to work on
> > very regular problems, like applying the same transformation at every
> > point in an array or something -- exactly like the vectorized simd
> > functions we all know and love -- fft, sum, etc, that run on a chip
> > designed to do the same thing to several datapoints all at once, not as
> > a loop on the cpu working one at a time. note how your program was
> > really three different programs.
> > > >
> > > > the toolbox gives you 'batch' for this:
> > > > http://www.mathworks.com/help/distcomp/batch.html
> > > >
> > > > and i don't see that it limits what you can do in each worker, so it
> > looks like you can go nuts running into nondeterministic concurrency
> > bugs. :)
> > > >
> > > > > Additionally, I can see parfor loops having some utility in video
> > > > > preloading, although I haven't given it a go myself.
> > > >
> > > > not sure what you mean by preloading (computing a complicated
> > stimulus offline? sure. but reading a huge stimulus from disk? won't
> > help.), but parfor requires the workers to be both pure and independent,
> > so it wouldn't work for the concurrency we've been discussing.
> > > >
> > > >
> > http://www.mathworks.com/help/distcomp/getting-started-with-parfor.html#\
> > brdqg4q-1
> > > >
> > > > -e
> > > >
> > >
> >
>