QUEST algorithm

1
Hello everyone,

I have a question related to the QUEST algorithm. I am measuring orientation discrimination thresholds in a 2-interval, 2AFC, same/different task.
After reading the Watson and Pelli 1983 paper and the documentation that ships with the psychtoolbox implementation of QUEST, I have incorporated the algorithm into my experiment code, such that each trial is assigned a difference value with QuestQuantile and then the difference is either implemented or not, assigned randomly (for "different" or "same" trials). Thereafter, q is updated with the intensity being set to the difference that would have been set, if the trial was a "different" trial.
Since I want to measure learning on this task, I don't want to train my subjects in performing the task very much before measuring their thresholds and don't want to extensively explore their thresholds, before getting a measurement. Thus, I want to get as open-ended an assessment of their initial thresholds as possible.
However, even when I set the initial value of tGuessSd to be rather large (on the order of 3 times as large as the initial value of tGuess), the staircase seems to converge rather quickly to rather small step sizes. This causes my estimates of the threshold to be very highly dependent on tGuess, which is exactly what I don't want.
Does anyone have any tips on how to use QUEST in such a manner that achieves rather stable estimates of the threshold, regardless of the initial tGuess. Do I have to crank up my initial value of tGuessSd even more? Is there another parameter I should be twiddling? I have not set any value for range - should I set that to some very large value?


Thanks a bunch,

Ariel

------------------------------------------------------------
Ariel Rokem
Helen Wills Neuroscience Institute
University of California, Berkeley
Barker Hall 132, MC #3190
Berkeley, CA 94720-3190
--
Tel: +1-510-6423134
Fax: +1-510-6435109
--
------------------------------------------------------------
"if it isn't worth doing, it isn't worth doing well." Donald Hebb

2
Hi Ariel,

I think you just need to convert to units that are more useful for your task.

For orientation discrimination say you start with an initial guess of -1 [ log10(.1)]

Prior standard deviation of 4.

Choose maximum orientation difference
maxTheta=45


trialTheta = QuestQuantile(q); %returns -.69
theta = maxTheta * 10^trialTheta
theta = min(theta, maxTheta);

This is your difference value +/- the standard orientation.

Sharon
--- In psychtoolbox@yahoogroups.com, Ariel Rokem <arokem@...> wrote:
>
> Hello everyone,
>
> I have a question related to the QUEST algorithm. I am measuring
> orientation discrimination thresholds in a 2-interval, 2AFC, same/
> different task.
> After reading the Watson and Pelli 1983 paper and the documentation
> that ships with the psychtoolbox implementation of QUEST, I have
> incorporated the algorithm into my experiment code, such that each
> trial is assigned a difference value with QuestQuantile and then the
> difference is either implemented or not, assigned randomly (for
> "different" or "same" trials). Thereafter, q is updated with the
> intensity being set to the difference that would have been set, if
> the trial was a "different" trial.
> Since I want to measure learning on this task, I don't want to train
> my subjects in performing the task very much before measuring their
> thresholds and don't want to extensively explore their thresholds,
> before getting a measurement. Thus, I want to get as open-ended an
> assessment of their initial thresholds as possible.
> However, even when I set the initial value of tGuessSd to be rather
> large (on the order of 3 times as large as the initial value of
> tGuess), the staircase seems to converge rather quickly to rather
> small step sizes. This causes my estimates of the threshold to be
> very highly dependent on tGuess, which is exactly what I don't want.
> Does anyone have any tips on how to use QUEST in such a manner that
> achieves rather stable estimates of the threshold, regardless of the
> initial tGuess. Do I have to crank up my initial value of tGuessSd
> even more? Is there another parameter I should be twiddling? I have
> not set any value for range - should I set that to some very large
> value?
>
>
> Thanks a bunch,
>
> Ariel
>
> ------------------------------------------------------------
> Ariel Rokem
> Helen Wills Neuroscience Institute
> University of California, Berkeley
> Barker Hall 132, MC #3190
> Berkeley, CA 94720-3190
> --
> Tel: +1-510-6423134
> Fax: +1-510-6435109
> --
> arokem@...
> http://socrates.berkeley.edu/~arokem/
> ------------------------------------------------------------
> "if it isn't worth doing, it isn't worth doing well." Donald Hebb
>
3
Hi Ariel,

I hope this answers your question. Quest is updated based on the response to the
trialTheta entry.

trialTheta = QuestQuantile(q);

% Update Quest and store data
q = QuestUpdate(q, trialTheta, response);

Update quest only when you have a valid response to the orientation difference that is
predicted as threshold by the quest procedure.


Often, depending upon your task your have to manipulate theta (value presented). For
example, for my current color saturation task. I need theta to go from .3 to 1. This is
how I do it.

output=10^trialTheta;
output=min(output,28);
stm.maxTheta=100;
thetai = (stm.maxTheta *1/output);
theta=(log10(thetai))*.4;
theta=abs(theta);

%set upper and lower bounds
theta=min(theta, 1);
theta=max(theta,.3);


Hope this helps.

Sharon Sally

--- In psychtoolbox@yahoogroups.com, Ariel Rokem <arokem@...> wrote:
>
> Hi -
>
> thanks a lot for the help - It works much better now!
>
> I have couple more questions. First of all, should the call to
> QuestUpdate be done with theta or with trialTheta (as the 'intensity'
> input variable)?
> Second, should the "same" trials differ from "different" trials in
> the way that QuestUpdate is called (or in a detection paradigm,
> "stimulus present" trials differ from "stimulus absent trials)?
> Should I call QuestUpdate with a value of zero (in the 'intensity'
> input variable) on "same" trials? I've tried doing that and gotten
> quite odd results and I was wondering what the general wisdom is on
> this point.
> I mean, the value of the intensity on trials when no stimulus is
> present is a virtual quantity. If the intensity is all the time
> changing, you can't expect the subjects to actually have some kind of
> idea of what intensity would be reasonable for a given trial, can
> you? Am I misunderstanding something essential?
>
> Cheers,
>
> Ariel
>
>
> On Mar 13, 2007, at 7:08 PM, salskippy wrote:
>
> > Hi Ariel,
> >
> > I think you just need to convert to units that are more useful for
> > your task.
> >
> > For orientation discrimination say you start with an initial guess
> > of -1 [ log10(.1)]
> >
> > Prior standard deviation of 4.
> >
> > Choose maximum orientation difference
> > maxTheta=45
> >
> > trialTheta = QuestQuantile(q); %returns -.69
> > theta = maxTheta * 10^trialTheta
> > theta = min(theta, maxTheta);
> >
> > This is your difference value +/- the standard orientation.
> >
> > Sharon
> > --- In psychtoolbox@yahoogroups.com, Ariel Rokem <arokem@> wrote:
> > >
> > > Hello everyone,
> > >
> > > I have a question related to the QUEST algorithm. I am measuring
> > > orientation discrimination thresholds in a 2-interval, 2AFC, same/
> > > different task.
> > > After reading the Watson and Pelli 1983 paper and the documentation
> > > that ships with the psychtoolbox implementation of QUEST, I have
> > > incorporated the algorithm into my experiment code, such that each
> > > trial is assigned a difference value with QuestQuantile and then the
> > > difference is either implemented or not, assigned randomly (for
> > > "different" or "same" trials). Thereafter, q is updated with the
> > > intensity being set to the difference that would have been set, if
> > > the trial was a "different" trial.
> > > Since I want to measure learning on this task, I don't want to train
> > > my subjects in performing the task very much before measuring their
> > > thresholds and don't want to extensively explore their thresholds,
> > > before getting a measurement. Thus, I want to get as open-ended an
> > > assessment of their initial thresholds as possible.
> > > However, even when I set the initial value of tGuessSd to be rather
> > > large (on the order of 3 times as large as the initial value of
> > > tGuess), the staircase seems to converge rather quickly to rather
> > > small step sizes. This causes my estimates of the threshold to be
> > > very highly dependent on tGuess, which is exactly what I don't want.
> > > Does anyone have any tips on how to use QUEST in such a manner that
> > > achieves rather stable estimates of the threshold, regardless of the
> > > initial tGuess. Do I have to crank up my initial value of tGuessSd
> > > even more? Is there another parameter I should be twiddling? I have
> > > not set any value for range - should I set that to some very large
> > > value?
> > >
> > >
> > > Thanks a bunch,
> > >
> > > Ariel
> > >
> > > ----------------------------------------------------------
> > > Ariel Rokem
> > > Helen Wills Neuroscience Institute
> > > University of California, Berkeley
> > > Barker Hall 132, MC #3190
> > > Berkeley, CA 94720-3190
> > > --
> > > Tel: +1-510-6423134
> > > Fax: +1-510-6435109
> > > --
> > > arokem@
> > > http://socrates.berkeley.edu/~arokem/
> > > ----------------------------------------------------------
> > > "if it isn't worth doing, it isn't worth doing well." Donald Hebb
> > >
> >
> >
> >
>
4
Hi Erik,

This is what you should do. Outside of your program just in the Matlab environment
create the Quest structure using Questcreate. Start with the input values from the demo.
Go through the steps: tTest = QuestQuantile(q);
%choose a response
% Update Quest and store data
q = QuestUpdate(q, trialTheta, response);
%Get the next test value, and so on.
This way you'll have a good understanding of how the algorithm is proceeding.
There is a lot of flexibility, but to begin with you'll find it easier to just convert the output
that QuestQuantile gives you (in your case output=10^tTest). So you can experiment with
different input values to Questcreate and also take a look at the simulation.

Best,

Sharon


--- In psychtoolbox@yahoogroups.com, "auda.tw" <audachang@...> wrote:
>
> Hi Sharon and Ariel,
>
> Do the threshold and sd have to be in log10 unit for the Quest
> algorithm to work properly?
>
> In my application, I would like to know the number of noise dots
> superimposed on a biological motion which makes the 2AFC performance
> at a 80% level. If I feed the actual number of dots into QuestUpdate,
> does QuestQuantile returns correct suggestion of the number of noise
> dots for the next trial?
>
> Thanks,
>
> Erik
>
> --- In psychtoolbox@yahoogroups.com, "salskippy" <ssally@> wrote:
> >
> > Hi Ariel,
> >
> > I hope this answers your question. Quest is updated based on the
> response to the
> > trialTheta entry.
> >
> > trialTheta = QuestQuantile(q);
> >
> > % Update Quest and store data
> > q = QuestUpdate(q, trialTheta, response);
> >
> > Update quest only when you have a valid response to the orientation
> difference that is
> > predicted as threshold by the quest procedure.
> >
> >
> > Often, depending upon your task your have to manipulate theta (value
> presented). For
> > example, for my current color saturation task. I need theta to go
> from .3 to 1. This is
> > how I do it.
> >
> > output=10^trialTheta;
> > output=min(output,28);
> > stm.maxTheta=100;
> > thetai = (stm.maxTheta *1/output);
> > theta=(log10(thetai))*.4;
> > theta=abs(theta);
> >
> > %set upper and lower bounds
> > theta=min(theta, 1);
> > theta=max(theta,.3);
> >
> >
> > Hope this helps.
> >
> > Sharon Sally
> >
> > --- In psychtoolbox@yahoogroups.com, Ariel Rokem <arokem@> wrote:
> > >
> > > Hi -
> > >
> > > thanks a lot for the help - It works much better now!
> > >
> > > I have couple more questions. First of all, should the call to
> > > QuestUpdate be done with theta or with trialTheta (as the
> 'intensity'
> > > input variable)?
> > > Second, should the "same" trials differ from "different" trials in
> > > the way that QuestUpdate is called (or in a detection paradigm,
> > > "stimulus present" trials differ from "stimulus absent trials)?
> > > Should I call QuestUpdate with a value of zero (in the 'intensity'
> > > input variable) on "same" trials? I've tried doing that and gotten
> > > quite odd results and I was wondering what the general wisdom is on
> > > this point.
> > > I mean, the value of the intensity on trials when no stimulus is
> > > present is a virtual quantity. If the intensity is all the time
> > > changing, you can't expect the subjects to actually have some kind
> of
> > > idea of what intensity would be reasonable for a given trial, can
> > > you? Am I misunderstanding something essential?
> > >
> > > Cheers,
> > >
> > > Ariel
> > >
> > >
> > > On Mar 13, 2007, at 7:08 PM, salskippy wrote:
> > >
> > > > Hi Ariel,
> > > >
> > > > I think you just need to convert to units that are more useful for
> > > > your task.
> > > >
> > > > For orientation discrimination say you start with an initial guess
> > > > of -1 [ log10(.1)]
> > > >
> > > > Prior standard deviation of 4.
> > > >
> > > > Choose maximum orientation difference
> > > > maxTheta=45
> > > >
> > > > trialTheta = QuestQuantile(q); %returns -.69
> > > > theta = maxTheta * 10^trialTheta
> > > > theta = min(theta, maxTheta);
> > > >
> > > > This is your difference value +/- the standard orientation.
> > > >
> > > > Sharon
> > > > --- In psychtoolbox@yahoogroups.com, Ariel Rokem <arokem@> wrote:
> > > > >
> > > > > Hello everyone,
> > > > >
> > > > > I have a question related to the QUEST algorithm. I am measuring
> > > > > orientation discrimination thresholds in a 2-interval, 2AFC, same/
> > > > > different task.
> > > > > After reading the Watson and Pelli 1983 paper and the
> documentation
> > > > > that ships with the psychtoolbox implementation of QUEST, I have
> > > > > incorporated the algorithm into my experiment code, such that each
> > > > > trial is assigned a difference value with QuestQuantile and
> then the
> > > > > difference is either implemented or not, assigned randomly (for
> > > > > "different" or "same" trials). Thereafter, q is updated with the
> > > > > intensity being set to the difference that would have been set, if
> > > > > the trial was a "different" trial.
> > > > > Since I want to measure learning on this task, I don't want to
> train
> > > > > my subjects in performing the task very much before measuring
> their
> > > > > thresholds and don't want to extensively explore their thresholds,
> > > > > before getting a measurement. Thus, I want to get as open-ended an
> > > > > assessment of their initial thresholds as possible.
> > > > > However, even when I set the initial value of tGuessSd to be
> rather
> > > > > large (on the order of 3 times as large as the initial value of
> > > > > tGuess), the staircase seems to converge rather quickly to rather
> > > > > small step sizes. This causes my estimates of the threshold to be
> > > > > very highly dependent on tGuess, which is exactly what I don't
> want.
> > > > > Does anyone have any tips on how to use QUEST in such a manner
> that
> > > > > achieves rather stable estimates of the threshold, regardless
> of the
> > > > > initial tGuess. Do I have to crank up my initial value of tGuessSd
> > > > > even more? Is there another parameter I should be twiddling? I
> have
> > > > > not set any value for range - should I set that to some very large
> > > > > value?
> > > > >
> > > > >
> > > > > Thanks a bunch,
> > > > >
> > > > > Ariel
> > > > >
> > > > > ----------------------------------------------------------
> > > > > Ariel Rokem
> > > > > Helen Wills Neuroscience Institute
> > > > > University of California, Berkeley
> > > > > Barker Hall 132, MC #3190
> > > > > Berkeley, CA 94720-3190
> > > > > --
> > > > > Tel: +1-510-6423134
> > > > > Fax: +1-510-6435109
> > > > > --
> > > > > arokem@
> > > > > http://socrates.berkeley.edu/~arokem/
> > > > > ----------------------------------------------------------
> > > > > "if it isn't worth doing, it isn't worth doing well." Donald Hebb
> > > > >
> > > >
> > > >
> > > >
> > >
> >
>