Hi, I’m trying to implement the QUEST procedure to estimate the threshold (~50% in this case) in my task. The task is a YES-NO detection task in which the subject has to detect the presence of a target.
The parameter that can vary during the experiment is the level of gaussian noise using the imnoise() function. In particular the standard deviation of the gaussian noise.
Basically this is a simplification of the experimental flow:
%% QUEST PARAMETERS
target_th = .51;
beta = 3.5;
delta = 0.5;
gamma = .5;
tguess = 0.15;
tguess_sd = 2;
max_noise = 1;
min_noise = 0;
step_noise = 0.001;
range_noise = max_noise - min_noise;
q = QuestCreate(tguess, tguess_sd, target_th, beta, delta, gamma, step_noise, range_noise);
%% STIMULUS
demoImageFolder = fileparts(which('cameraman.tif'));
im = imread(fullfile(demoImageFolder, 'cameraman.tif'));
%% EXPERIMENT
for i = 1:40
% Suggest value
SD = QuestQuantile(q);
% Create the image
im = imnoise(im, 'Gaussian', 0, SD);
% Collect the response
resp = randi([0 1]);
% Recompute
q = QuestUpdate(q, SD, resp);
end
% Final Estimate
QuestMean(q);
Now I have some problems regards:
- My variable (SD of noise) ranges between 0 and +Inf but sometimes the QUEST suggest me some negative values
- As described in the help section, the QUEST is based on the contrast concept so the y value (proportion of yes response in this case) should increase according to increasing contrast. However, in this case, the lowest level of SD should be associated with the lowest visibility.
- The final estimation (using
questmean() or questmode()) should be related to my target_th value. If I want another point on the psychometric function, do I simply need to type QuestQuantile(q, my_quantile)?
I’ve tried different solutions, but I’m not an expert in psychophysics:
- Putting a negative
beta for changing the weibull shape and using 1 - SD to revert the relationship between noise and yes-detection
- Transforming my SD value in log10(SD) for the QUEST and than reverting the value for the imnoise() function
The final estimation don’t really convince me so I’m wondering if I’m doing all correctly.
Thanks!
I’ve never used the QUEST implementation in PTB. But I can highly recommend Palamedes for a comprehensive range of methods from the simplest staircase through to the most recent adaptive Bayesian methods that handle all psychometric curve parameters. All psychometric functions include log and linear variants.
http://www.palamedestoolbox.org
The documentation is superb, as Palamedes is accompanied by a great text book that details each method, practically and then computationally. Lots of demos and online FAQ available if you can’t buy the book…
Kingdom, F.A.A. & Prins, N. (2016) Psychophysics: A Practical Introduction, 2nd edition. Academic Press: an imprint of Elsevier, London
Thanks for the suggestion! I’ve read the book and is amazing! I’ll try the toolbox!
I have another bit of general advice, concerning the choice of method: Don’t use the Yes-No method! According to signal detection theory, (SDT; which is really the only game in town as far as psychophysical detection experiments are concerned) the percentage of ‘Yes’ responses you get (the Hit rate) depends on two things: the way internal response magnitude depends on signal strength, and the subjective criterion level of internal response adopted by the observer for saying ‘Yes’. Usually you want to know about the first of these, while discounting or holding constant the second. These two factors can, in principle, be disentangled by also recording the false alarm rate on trials where no signal is presented (FA rate; proportion of Yes responses on non-signal trials). But what you do after that to combine these 2 response probabilities is tricky, liable to be rather ‘noisy’ (because estimating FA rates from low probabilities is inherently unreliable…), does require a fairly good knowledge of SDT, and is not (as far as I can see) amenable to the QUEST method. The most obvious solution is to cast your experiment in the form of a 2-alternative forced-choice design (2AFC), where (to a large extent) the problems just mentioned go away, or are minimised. But remember that in 2AFC the two alternative stimuli (S,N) must both be presented on each trial, with the observer saying which of 2 stimulus positions in space, or stimulus intervals in time, contained the signal S. Take a look at Kingdom & Prins’ textbook, or Macmillan & Creelman’s for more on all this. I hope this helps.
Mark
Thanks for your detailed response. If I understand correctly you are suggesting to use a 2AFC to control for the FA rate. I’ve discussed a lot about this point with my colleagues. Basically I need to have this threshold because I want to use the threshold stimulus in another experiment where I need to have roughly 50% of detected stimulus and 50% of non detected stimulus in my experimental task. In terms of SDT I’m interested in having 50% of hit and 50% of miss. A lot of paper in my field (visual consciousness) use the yes-no but inside the psychophysical literature this is often discouraged. If I get the point of several papers that I’ve read, the 50% threshold could be not estimated with 2AFC given that the usual threshold is 75%. If I use the 75% stimulus in a simple detection task i should get more hit than miss.
OK, I understand better now what you are aiming for. But again , I’d like to offer some words of caution, based on SDT. First, in SDT there is no threshold that divides seen from not-seen. SDT emphasises that we can tease apart sensitivity/discriminability from response-criterion setting - but we can’t say whether things are ‘seen’ or not. All we know is how the system performs. Think of blindsight - visual performance without visual consciousness. All stimuli are held to create some sensory response, even if a very weak one. The practical consequence is this: we can define a notional threshold, such as d’=1 (76% correct in 2AFC), but this simply defines a certain level of weak detectability, not a boundary between seen and not-seen. And importantly for you, in the yes-no task, the 50% “threshold” also needs to be understood differently: the stimulus that evokes 50% Yes responses is the stimulus whose internal representation aligns with the observer’s response criterion. [In this case, the internal response distribution straddles the response criterion, and if the median of this distribution aligns with the criterion level, then 50% of responses are above criterion, hence 50% Yes]. So, in principle, the stimulus giving you 50% Yes tells you about the criterion, and not about sensitivity or detectability. This point has not been sufficiently recognised in the literature, I feel. But how many of these Yes trials represent ‘seen’ is unknown. Because ‘seen’ is not a concept that forms part of SDT. Now, here’s an interesting possibility: that observers are able to set their criterion in the Yes-No task in order to maintain some limited FA rate, such as 5%. If they did that (eg using feedback about errors in the task) then we can very easily compute the value of d’ for the stimulus that gives 50% Yes. For 2% false alarms d’ would be 2.05; 5% false alarms d’ = 1.64; for 15.9% FA, d’ = 1 for the stimulus giving 50% Hit rate. So you can see that modest variations in the FA rate lead to substantial variations in performance (d’) at 50% Yes. It seems uncontroversial to suppose that higher d’ values lead to more and better ‘seeing’, but there is little reason to think that 50% Yes corresponds to some fixed rate of ‘seeing’. Revealing consciousness by experiments sure is a hard problem ! I hope this helps a bit.
Thanks! I’ve realized that is a very complicated (but intriguing) topic. I’ve to think about that point however you are reasoning more about SDT than in terms of seen/not seen. A d’ greater than 0 (regardless the amount of FA) says that the system is detecting something. So in this way I have two possibilites:
- Control the FA rate (as you have suggested)
- Find a method, if exists, to use an adaptive procedure based on SDT and d’ instead of proportion of yes responses?
is it correct?