Using QUEST for psychometric function of discrimination task?

1
Hi all,

I'm testing observers' slant discrimination using 2IFC. The context of the experiment requires fast estimation of the psychometric function, so rather than using method of constant stimuli I've been advised to use multiple interleaved QUEST staircases. I set each staircase to converge at a different point on the psychometric function and then either use the 'threshold estimate' from each staircase, or bin the data before fitting a psychometric function to it. Also, the pdf for all staircases are updated on each trial, thus speeding up convergence on their respective targets.

What I've done so far is to use separate runs to test either side of the base slant (60°): in one run all test slants are larger than the base slant and in the other run they are smaller than the base slant. (Obviously the order of test and base slant presentation is randomized).

This results in 2 separate sets of data that can be plotted as 'probability of a correct response' against 'slant difference', and fitted with psychometric functions that asymptote at y = 0.5 and y = 1.

However, what I would like to plot is 'probability of perceiving test slant as larger than base slant' against 'test slant', which would allow me to fit a single psychometric function to all of the data. i.e. it would asymptote at y = 0 and y = 1.

Here are some crude illustrations of what I'm talking about:
http://imageshack.us/photo/my-images/263/questtopsychometric.png/

Is this possible using QUEST?

Many thanks,
Aidan Murphy (Binocular Vision Lab, University of Birmingham, UK)
2
Hi Diederick,

many thanks for your reply. The observers' task is to indicate which of the 2 intervals contained the smaller slant - the first of the second.

I have tried transforming the signal intensity range from 0 to 1 (which is what QUEST works with) to say, 40° to 80° (which is an appropriate range of test slants, about the 60° base slant). However, the problem is that QUEST only cares about the accuracy of the observer's response, and updates the pdf based on this information. Therefore, we expect a high probability of correct responses when signal intensity is 0 or 1, and 0.5 correct in the middle (i.e. when test slant = base slant). As far as I know, there is no combination of parameters that I can provide to a single QUEST staircase to generate such a fit, therefore I split the session into 2.

As you suggested, I can recode data from each trial to find the proportion of responses where the test slant was judged to be greater than the base slant. If I do this, I assume I must bin the raw data in order to fit a psychometric and disregard QUEST's threshold estimates, since they're based on response accuracy?

This seems like the best solution, but it still seems to require separating staircases that test test slants greater than the abse slant from those testing test slants smaller than the abse slant. Is this correct?

Many thanks for your help.
Aidan


--- In psychtoolbox@yahoogroups.com, "Diederick C. Niehorster" <dcnieho@...> wrote:
>
> Dear Aidan,
>
> On Fri, Mar 30, 2012 at 21:11, apm909 <aidanmurphy1@...> wrote:
>
> > What I've done so far is to use separate runs to test either side of the
> > base slant (60°): in one run all test slants are larger than the base slant
> > and in the other run they are smaller than the base slant. (Obviously the
> > order of test and base slant presentation is randomized).
> >
> > This results in 2 separate sets of data that can be plotted as
> > 'probability of a correct response' against 'slant difference', and fitted
> > with psychometric functions that asymptote at y = 0.5 and y = 1.
> >
>
> Some crucial information about your task is missing from your story, that
> is, what is the task your observers performed. Did they press one key if
> they saw the slant as smaller than the pedestal slant, and another if
> larger? Then you should be able to fit a psychometric function to this data
> directly. If they did a different task, could you recode the responses to
> yield this?
>
> That said, why did you split up your staircases like this? In my
> understanding. If you want to avoid problems of hysteresis in the response
> strategy, you can simply interleave two identical staircases covering the
> whole range from smaller than base slant to larger than base slant (and
> then combine the data straightforwardly in analysis). However, if you are
> interested in observer's threshold, your tasks makes sense to me (an
> adaptive staircase's estimate of the midpoint of the psychometric function
> converges much faster than itys estimate of the slope).
>
> One disclaimer: I'm not terribly experienced in this stuff, having only run
> pretty basic discrimination tasks myself. So if somebody with more
> experience finds a whole in my story, please do educate us!
>
> Hope this is of some help, sorry if it isn't.
>
> Best,
> Dee
>
3
Hi Aidan,
I have never used QUEST myself, but i would find it weird if you can only feed correct or not to QUEST. I assume one of the intervals always contained the base 60° slant? You could then probably also feed QUEST something like 0=slant of interval under staircase control is smaller than base slant, 1=slant is larger, instead of correct or not (which would/might require some recoding of the response in your code). I'd be very surprised if that is not possible. Note that with what I'm suggesting, very small testing slants would (nearly) always elicit a 0 (smaller) response, very large slants a 1 (larger) response, and slants equal to the base slant on average a 0.5 response (hopefully).
If this really turns out to be impossible, you could have a look at MinExpEntStair (and MinExpEntStairDemo). It is my and Jeff Saunder's implementation of a method similar to Kontsevich and Tyler's (1999) adaptive staircase. I'm not sure if that would be easier for you, but it'll eat any range of possible probe values that you throw at it. Note that I'll be on holiday for two weeks starting very soon, so I wont be able to help you through getting it set up I'm afraid. I'd go down the route of trying to get QUEST to work first, simply because it seems your almost there.
Lastly, after recoding your current data you would not have to bin it or anything first. you should be able to fit a psychometric function to it directly, using standard methods.
Hope that helps,
Dee
On Fri, Mar 30, 2012 at 22:10, apm909 <aidanmurphy1@...> wrote:

Hi Diederick,

many thanks for your reply. The observers' task is to indicate which of the 2 intervals contained the smaller slant - the first of the second.

I have tried transforming the signal intensity range from 0 to 1 (which is what QUEST works with) to say, 40° to 80° (which is an appropriate range of test slants, about the 60° base slant). However, the problem is that QUEST only cares about the accuracy of the observer's response, and updates the pdf based on this information. Therefore, we expect a high probability of correct responses when signal intensity is 0 or 1, and 0.5 correct in the middle (i.e. when test slant = base slant). As far as I know, there is no combination of parameters that I can provide to a single QUEST staircase to generate such a fit, therefore I split the session into 2.

As you suggested, I can recode data from each trial to find the proportion of responses where the test slant was judged to be greater than the base slant. If I do this, I assume I must bin the raw data in order to fit a psychometric and disregard QUEST's threshold estimates, since they're based on response accuracy?

This seems like the best solution, but it still seems to require separating staircases that test test slants greater than the abse slant from those testing test slants smaller than the abse slant. Is this correct?

Many thanks for your help.
Aidan



--- In psychtoolbox@yahoogroups.com, "Diederick C. Niehorster" <dcnieho@...> wrote:
>
> Dear Aidan,
>
> On Fri, Mar 30, 2012 at 21:11, apm909 <aidanmurphy1@...> wrote:
>
> > What I've done so far is to use separate runs to test either side of the
> > base slant (60°): in one run all test slants are larger than the base slant

> > and in the other run they are smaller than the base slant. (Obviously the
> > order of test and base slant presentation is randomized).
> >
> > This results in 2 separate sets of data that can be plotted as
> > 'probability of a correct response' against 'slant difference', and fitted
> > with psychometric functions that asymptote at y = 0.5 and y = 1.
> >
>
> Some crucial information about your task is missing from your story, that
> is, what is the task your observers performed. Did they press one key if
> they saw the slant as smaller than the pedestal slant, and another if
> larger? Then you should be able to fit a psychometric function to this data
> directly. If they did a different task, could you recode the responses to
> yield this?
>
> That said, why did you split up your staircases like this? In my
> understanding. If you want to avoid problems of hysteresis in the response
> strategy, you can simply interleave two identical staircases covering the
> whole range from smaller than base slant to larger than base slant (and
> then combine the data straightforwardly in analysis). However, if you are
> interested in observer's threshold, your tasks makes sense to me (an
> adaptive staircase's estimate of the midpoint of the psychometric function
> converges much faster than itys estimate of the slope).
>
> One disclaimer: I'm not terribly experienced in this stuff, having only run
> pretty basic discrimination tasks myself. So if somebody with more
> experience finds a whole in my story, please do educate us!
>
> Hope this is of some help, sorry if it isn't.
>
> Best,
> Dee
>


4
Ok, I think I understand now! So long as I have informed QUEST that the probability of a '1' response when the signal intensity is -infinity (i.e. the gamma parameter), it doesn't matter whether the response represents some contingency other than accuracy.

I would imagine that you're probably right that QUEST can handle this, so I'll stick with it before trying MinExpEntStair. Much of my experiment is based on a Knill & Saunders paper that didn't really go into detail about the staircase procedure, so it's nice to know what methods others have used.

Thanks again Diederick, and enjoy your holiday!
Aidan

--- In psychtoolbox@yahoogroups.com, "Diederick C. Niehorster" <dcnieho@...> wrote:
>
> Hi Aidan,
>
> I have never used QUEST myself, but i would find it weird if you can only
> feed correct or not to QUEST. I assume one of the intervals always
> contained the base 60° slant? You could then probably also feed QUEST
> something like 0=slant of interval under staircase control is smaller than
> base slant, 1=slant is larger, instead of correct or not (which
> would/might require some recoding of the response in your code). I'd be
> very surprised if that is not possible. Note that with what I'm suggesting,
> very small testing slants would (nearly) always elicit a 0 (smaller)
> response, very large slants a 1 (larger) response, and slants equal to the
> base slant on average a 0.5 response (hopefully).
>
> If this really turns out to be impossible, you could have a look at
> MinExpEntStair (and MinExpEntStairDemo). It is my and Jeff Saunder's
> implementation of a method similar to Kontsevich and Tyler's (1999)
> adaptive staircase. I'm not sure if that would be easier for you, but it'll
> eat any range of possible probe values that you throw at it. Note that I'll
> be on holiday for two weeks starting very soon, so I wont be able to help
> you through getting it set up I'm afraid. I'd go down the route of trying
> to get QUEST to work first, simply because it seems your almost there.
>
> Lastly, after recoding your current data you would not have to bin it or
> anything first. you should be able to fit a psychometric function to it
> directly, using standard methods.
> Hope that helps,
> Dee
> On Fri, Mar 30, 2012 at 22:10, apm909 <aidanmurphy1@...> wrote:
>
> > **
> >
> >
> > Hi Diederick,
> >
> > many thanks for your reply. The observers' task is to indicate which of
> > the 2 intervals contained the smaller slant - the first of the second.
> >
> > I have tried transforming the signal intensity range from 0 to 1 (which is
> > what QUEST works with) to say, 40° to 80° (which is an appropriate range of
> > test slants, about the 60° base slant). However, the problem is that QUEST
> > only cares about the accuracy of the observer's response, and updates the
> > pdf based on this information. Therefore, we expect a high probability of
> > correct responses when signal intensity is 0 or 1, and 0.5 correct in the
> > middle (i.e. when test slant = base slant). As far as I know, there is no
> > combination of parameters that I can provide to a single QUEST staircase to
> > generate such a fit, therefore I split the session into 2.
> >
> > As you suggested, I can recode data from each trial to find the proportion
> > of responses where the test slant was judged to be greater than the base
> > slant. If I do this, I assume I must bin the raw data in order to fit a
> > psychometric and disregard QUEST's threshold estimates, since they're based
> > on response accuracy?
> >
> > This seems like the best solution, but it still seems to require
> > separating staircases that test test slants greater than the abse slant
> > from those testing test slants smaller than the abse slant. Is this correct?
> >
> > Many thanks for your help.
> > Aidan
> >
> >
> > --- In psychtoolbox@yahoogroups.com, "Diederick C. Niehorster" <dcnieho@>
> > wrote:
> > >
> > > Dear Aidan,
> > >
> > > On Fri, Mar 30, 2012 at 21:11, apm909 <aidanmurphy1@> wrote:
> > >
> > > > What I've done so far is to use separate runs to test either side of
> > the
> > > > base slant (60°): in one run all test slants are larger than the base
> > slant
> >
> > > > and in the other run they are smaller than the base slant. (Obviously
> > the
> > > > order of test and base slant presentation is randomized).
> > > >
> > > > This results in 2 separate sets of data that can be plotted as
> > > > 'probability of a correct response' against 'slant difference', and
> > fitted
> > > > with psychometric functions that asymptote at y = 0.5 and y = 1.
> > > >
> > >
> > > Some crucial information about your task is missing from your story, that
> > > is, what is the task your observers performed. Did they press one key if
> > > they saw the slant as smaller than the pedestal slant, and another if
> > > larger? Then you should be able to fit a psychometric function to this
> > data
> > > directly. If they did a different task, could you recode the responses to
> > > yield this?
> > >
> > > That said, why did you split up your staircases like this? In my
> > > understanding. If you want to avoid problems of hysteresis in the
> > response
> > > strategy, you can simply interleave two identical staircases covering the
> > > whole range from smaller than base slant to larger than base slant (and
> > > then combine the data straightforwardly in analysis). However, if you are
> > > interested in observer's threshold, your tasks makes sense to me (an
> > > adaptive staircase's estimate of the midpoint of the psychometric
> > function
> > > converges much faster than itys estimate of the slope).
> > >
> > > One disclaimer: I'm not terribly experienced in this stuff, having only
> > run
> > > pretty basic discrimination tasks myself. So if somebody with more
> > > experience finds a whole in my story, please do educate us!
> > >
> > > Hope this is of some help, sorry if it isn't.
> > >
> > > Best,
> > > Dee
> > >
> >
> >
> >
>