Documentation for Quest Procedure?

1
hi friends,

can anyone point me to documentation for the quest procedure? i can't seem to
find any anywhere, and the demo program that comes with it isn't particularly
informative. in particular, my initial attempts at using the procedure always
result in quest estimating the threshold downward, and i need it to estimate
upward.

also, something i read somewhere said that the initial guess and s.d. of the
guess need to be fairly accurate, as the procedure will have difficulty
generating an accurate estimate if the true threshold is more than one
s.d. away from the initial guess. despite this, the procedure doesn't seem to
ever guess very far away from my initial guess. for example, true threshold
is 40, i input an initial guess of 30 and s.d. of 10 (and even 20), and quest
never guesses beyond 32.5 (i actually have to guess -30; otherwise quest
guesses downward, as i already mentioned: i need to figure out how to get it
guess upward). so, how can the procedure to guess "more boldly" (so to
speak)? i've tried increasing beta (the "steepness" parameter) from 3.5 to
7.0, but this didn't help.

so, as i said, i'm looking for some good documentation, and/or some good,
complete, real examples. thanks in advance for any and all assistance!

cheers,
doug
2
dear doug

i suggest you read the published article:
Watson, A. B. & Pelli, D. G. (1983) QUEST: a Bayesian adaptive psychometric method. Percept Psychophys, 33 (2), 113-20.

which you can download as a PDF from here:

the help text that comes with the demo suggests that the "intensity" scale you provide to QUEST be a log10 of the physical intensity (eg contrast). thus the values you are using are quite extreme, corresponding to a physical range of 20 log units. the algorithm scales well, but there are some compromises necessary to implement a computer program with finite resources. in the case you describe, i think you will get the behavior you expect if you provide QUEST with log10(30), log10(40), etc.  i think this will all become obvious to you once you read the article and use a more reasonable range.

hope that helps

best

denis

Denis Pelli
Professor of Psychology and Neural Science
New York University



On Mar 16, 2008, at 1:19 AM, Doug Morse wrote:

hi friends,

can anyone point me to documentation for the quest procedure? i can't seem to
find any anywhere, and the demo program that comes with it isn't particularly
informative. in particular, my initial attempts at using the procedure always
result in quest estimating the threshold downward, and i need it to estimate
upward. 

also, something i read somewhere said that the initial guess and s.d. of the
guess need to be fairly accurate, as the procedure will have difficulty
generating an accurate estimate if the true threshold is more than one
s.d. away from the initial guess. despite this, the procedure doesn't seem to
ever guess very far away from my initial guess. for example, true threshold
is 40, i input an initial guess of 30 and s.d. of 10 (and even 20), and quest
never guesses beyond 32.5 (i actually have to guess -30; otherwise quest
guesses downward, as i already mentioned: i need to figure out how to get it
guess upward). so, how can the procedure to guess "more boldly" (so to
speak)? i've tried increasing beta (the "steepness" parameter) from 3.5 to
7.0, but this didn't help.

so, as i said, i'm looking for some good documentation, and/or some good,
complete, real examples. thanks in advance for any and all assistance!

cheers,
doug


3
hi dennis,

thanks so much for your input! i've read your 1983 article several times now
and doing so has been quite helpful.

i figured out the problem with the python implemetation not going past 2.5 log
units from the initial guess in its estimation: there is a "range" optional
parameter, which defaults to 5, that specifies the size of the table. as soon
as i set it to a larger value, the implementation will estimate beyond 2.5 log
units and seems to behave more as i would expect.

the one thing i still can't figure out is how one specifies in which direction
the procedure estimates (i.e., above or below the initial guess). i could not
discern this from the 1983 article (but perhaps i missed something?). in my
testing, the procedure always estimates downward, no matter what parameters i
specify. what's interesting is that the included demo *does* estimate upward,
but for the life of i me i cannot figure why/how nor how to replicate.

so, i loathe to trouble you again, but if you have any insights into how i can
specify that the procedure estimate upward, i would be most grateful. this is
the only issue i have at present that's keeping me from finishing my software
and start data collection.

most gratefully,
doug


On Sun, Mar 16, 2008 at 01:55:09AM -0400, Denis Pelli wrote:
> dear doug
>
> i suggest you read the published article:
> Watson, A. B. & Pelli, D. G. (1983) QUEST: a Bayesian adaptive
> psychometric method. Percept Psychophys, 33 (2), 113-20.
>
> which you can download as a PDF from here:
> http://www.psych.nyu.edu/pelli/papers.html#1983
>
> the help text that comes with the demo suggests that the "intensity"
> scale you provide to QUEST be a log10 of the physical intensity (eg
> contrast). thus the values you are using are quite extreme,
> corresponding to a physical range of 20 log units. the algorithm
> scales well, but there are some compromises necessary to implement a
> computer program with finite resources. in the case you describe, i
> think you will get the behavior you expect if you provide QUEST with
> log10(30), log10(40), etc. i think this will all become obvious to
> you once you read the article and use a more reasonable range.
>
> hope that helps
>
> best
>
> denis
>
> Denis Pelli
> Professor of Psychology and Neural Science
> New York University
> http://psych.nyu.edu/pelli/