Re: Video synchronization

1
>Dear Dr. Brainard and Dr. Pelli,
>
>My name is Martin Linenweber. I am a graduate student in Computer
>Science at Washington University at St. Louis. Previously I worked
>for Maurizio Corbetta and Gordon Shulman in the NeuroImaging
>Laboratory at Washington University's Medical School. I have recently
>been hired back to do some Macintosh programming for them.
>
>One of the problems to solve is the synchronization of the display
>associated with newer hardware (G3's/G4's). We have been able to
>avoid this problem by using an older PowerMac, but this is no longer
>viable.
>
>In looking through your PsychToolBox routines, we were excited to see
>that someone else had thought about and addressed this problem. While
>some people in our lab are going to use Matlab and PsychToolBox,
>others prefer to use our in house suite of functions as well as the
>manner in which they are used to working. This involves writing code
>in C through CodeWarrior and using the Macintosh functions such as
>AnimatePalette and AnimateEntry for display. To this end, I have been
>charged with duplicating the success of PsychToolBox by taking what
>you have done with and adapting it to our needs in a "pure C
>environment" (i.e. without Matlab references).
>
>It is my understanding that PsychToolBox is situated between Matlab
>and VideoToolBox. Not only does it perform as a gateway but also does
>other work, some of which is associated with the specific problem
>above. We have come up with three options to do the adaptation:

I think this basic view is tenable for your purposes. All of the
really hard work dealing with the video drivers and getting around
the Mac OS's desire to modify
lookup tables for you is done by the video toolbox. The C code for the
Psychtoolbox routine SCREEN does essentially two things: a) it interfaces
to the MATLAB MEX file API, reading the arguments MATLAB passes, massaging
them, etc. and similarly on return to MATLAB. b) it encapsulates calls
to the VideoToolbox routines and thus provides a conceptually simpler interface
to the hardware. There are very few cases where you need lower level access
to the VideoToolbox than provided by the SCREEN functions.

> 1) What would be ideal (from a coding standpoint) is if I
>could use PsychToolBox without any modifications, by adding some sort
>of wrapper to it. That is, passing in arrays with the appropriate
>data. This doesn't look feasible since PsychToolBox operates on
>Matlab structures.
> 2) The other closely related option is to artificially create
>these Matlab structures, filling them with appropriate information.
>This seems to be rather difficult and too round about.
> 3) The third option is to remove all references to Matlab from
>PsychToolBox and use the extra work (e.g. testing the MaxPriority).
>This is probably the most laborious, but could also be done on a case
>by case basis depending on what functionality we need. I have done
>this for a few routines associated with the VBL problem. Aside from
>the number of routines to change, there seems to be some difficulty in
>removing them from there current environment. For example, memory
>allocation and removal of interrupt routines don't seem to be properly
>taken care of (I can be more specific if needed).
>
>If you could comment on the validity of these options (as well as the
>overall goal), it would be greatly appreciated.

I think the most straightforward way to use our code is to add C wrappers
that allow you to call directly into SCREEN as is, your option 2. I agree
that your option 1 will end up being option 2 by the time you're done --
the only way I see to add wrappers is to fill in the appropriate MATLAB
structures.

The advantage of this is that once it was all in place, you could
take advantage of any future
bug fixes or enhancements we make to our code, and also port your stuff
to Windows as we get our Windows interface working better. In addition,
you wouldn't then have to worry about dissecting our code into the parts
that actually do the work and the parts that are the interface to the MATLAB
API. If we were starting from scratch now, we would have structured the
C code more modularly, so that this distinction was clean. But the fact is
that it is a bit tangled in the current implementation.

I believe that the Mathworks distributes a library designed to let you
call MATLAB from C, and I would imagine that this library would provide
the code you needed to set up MATLAB structures in C and then call our
routines directly. But I've never tried it and this could recollection
could be incorrect.

The disadvantage of this approach is that to make the interface work, you'll
need to translate a lot of fundamentally integer data into MATLAB's double
precision format, after which our code will translate it back to integers
again. Fundamentally not very elegant or efficient. I think the efficiency
is not really an issue, given the speed of modern computers. But it might
be that it is tricky to make it all work robustly. You might try this
approach for a week and see how easy it is once you get into it. If not
easy, give up and go to option 3.

Certainly it would be possible to pull out the relevant bits of our code and
use them directly (your option 3). I think what you might do here is just
use the SCREEN code as a template for the appropriate calls into the
VideoToolbox and not worry about using our code per se.

I would stay away from trying to use AnimatePalette and AnimateEntry.
It's because it is difficult (if not impossible) to get those to really
do what you want that the VideoToolbox goes directly to the VideoDrivers.
Think of the VideoToolbox as a psychophysicists' substitute for those
routines.

>Also, in my introduction I mentioned how our VBL problem has been
>lingering for several years. In this time we have not addressed the
>problem at all. In the past few days I wanted to implement my partial
>solution (from option 3), but when I went to replicate the problem, it
>did not present itself. The test essentially ran AnimatePalette a
>large number of times. This loop was timed with the getTicks()
>routine. The display ran smoothly and the time duration was correct.
>This produced mixed feeling and more questions. Does this problem
>still exist? Is our test flawed? Is something more devious
>happening?
>
>We have been looking for the test programs written years ago, which
>back then showed the VBL problem. If you have any test program which
>demonstrate the problem we would appreciate the code or a description
>of such code.

The VideoToolbox distribution comes with a test program, TimeVideo,
which will exercise the video driver and give you a report on its
performance. Very similar functionality is provided from within
MATLAB by the Psychtoolbox program ScreenTest.

GetTicks is not a reliable way to time things to high precision,
although it will give you a ballpark estimate. I forget the names
of the better low level calls, but I think they are mentioned in
the help for the MATLAB GetSecs MEX file (type "help GetSecs"
at the MATLAB prompt.)

Best,

David
2
Dear Drs. Pelli and Brainard,

Thank you both for responding so quickly.

As per your suggestions
I will look into wrapping the Psychtoolbox functions
and will try the TimeVideo test program.


As you may recall I wanted to move Psychtoolbox from a Matlab environment to
a stand-alone C one. The following addresses the points I had made about
removing Matlab references from the Psychtoolbox routines. Dr. Pelli's
response follows.

>>> Aside from
>>> the number of routines to change, there seems to be some difficulty in
>>> removing them from there current environment. For example, memory
>>> allocation and removal of interrupt routines don't seem to be properly
>>> taken care of (I can be more specific if needed).
>
> I am puzzled by this remark. We have taken some care to follow all the
> relevant apple rules to create a code base that will be as reliable as
> possible for experiments. And indeed the psychtoolbox code does operate
> very reliably, as shown by our many rigorous test programs and by user
> reports. Could you please write to me with specifics?



What I wanted to do was to make calls such as SetClutDriverWaitsForBlanking
and others which are in GetScreenInfo.c before running our normal program.
I modified GetScreenInfo.c as well as VBL.c. I have attached a copy of both
modifed files.

The test simply entailed the following bits of code:
device=GetScreenDevice(0);
SetClutDriverWaitsForBlanking(device);
// the rest of the program would then follow, but for testing
// purposes the program exits here.

The one reoccurring changes I made was to remove reference to PsychTable.
The only times it was used here was to obtain priority references therefore:
psychTable->priority was replaced with getPriority()
SetPsychPriority was replaced with SetPriority()
GetPsychTable() was commented out.

Other references to Matlab (e.g. IsMATLABBackgroundingEnabled) were
commented out as well.


The two problems I had are as follows:
1) In GetScreenInfo the pointer screenInfo doesn't get allocated. In my
code, you will see I use the firstTime variable for this purpose.
Line 91 is were I do the alllocation. Perhaps this allocation happens
somewhere else.

2) In VBL.c the routine PeekBlankingTime calls InstallVBL(device) but
doesn't call RemoveVBL(device). This doesn't seem to pose a problem until
the program exits. At this time CodeWarrior crashes and I have to reboot.
My solution was to call RemoveVBL before exiting PeekBlankingTime (Line
205). Alternatively, one can call it before the main program exits.



Thank You
Martin Linenweber