fitConf {statConfR} | R Documentation |
Fit a static confidence model to data
Description
This function fits one static model of decision confidence to binary choices and confidence judgments. It calls a corresponding fitting function for the selected model.
Usage
fitConf(data, model = "SDT", nInits = 5, nRestart = 4)
Arguments
data |
a
|
model |
|
nInits |
|
nRestart |
|
Details
The fitting routine first performs a coarse grid search to find promising
starting values for the maximum likelihood optimization procedure. Then the best nInits
parameter sets found by the grid search are used as the initial values for separate
runs of the Nelder-Mead algorithm implemented in optim
.
Each run is restarted nRestart
times.
Mathematical description of models
The computational models are all based on signal detection theory (Green & Swets, 1966). It is assumed
that participants select a binary discrimination response about a stimulus
.
Both
and
can be either -1 or 1.
is considered correct if
.
In addition, we assume that there are
different levels of stimulus discriminability
in the experiment, i.e. a physical variable that makes the discrimination task easier or harder.
For each level of discriminability, the function fits a different discrimination
sensitivity parameter
. If there is more than one sensitivity parameter,
we assume that the sensitivity parameters are ordered such as
.
The models assume that the stimulus generates normally distributed sensory evidence
with mean
and variance of 1. The sensory evidence
is compared to a decision
criterion
to generate a discrimination response
, which is 1, if
exceeds
and -1 else.
To generate confidence, it is assumed that the confidence variable
is compared to another
set of criteria
, depending on the
discrimination response
to produce a
-step discrete confidence response.
The number of thresholds will be inferred from the number of steps in the
rating
column of data
. Thus, the parameters shared between all models are:
sensitivity parameters
,...,
(
: number of difficulty levels)
decision criterion
confidence criterion
,
, ...,
,
,
,...,
(
: number of confidence categories available for confidence ratings)
How the confidence variable is computed varies across the different models.
The following models have been implemented so far:
Signal Detection Rating Model (SDT)
According to SDT, the same sample of sensory
evidence is used to generate response and confidence, i.e.,
and the confidence criteria span from the left and
right side of the decision criterion
(Green & Swets, 1966).
Gaussian Noise Model (GN)
According to the model, is subject to
additive noise and assumed to be normally distributed around the decision
evidence value
with a standard deviation
(Maniscalco & Lau, 2016).
The parameter
is a free parameter.
Weighted Evidence and Visibility model (WEV)
WEV assumes that the observer combines evidence about decision-relevant features
of the stimulus with the strength of evidence about choice-irrelevant features
to generate confidence (Rausch et al., 2018). Here, we use the version of the WEV model
used by Rausch et al. (2023), which assumes that is normally
distributed with a mean of
and standard deviation
.
The parameter
quantifies the amount of unsystematic variability
contributing to confidence judgments but not to the discrimination judgments.
The parameter
represents the weight that is put on the choice-irrelevant
features in the confidence judgment.
and
are fitted in
addition to the set of shared parameters.
Post-decisional accumulation model (PDA)
PDA represents the idea of on-going information accumulation after the
discrimination choice (Rausch et al., 2018). The parameter indicates the amount of additional
accumulation. The confidence variable is normally distributed with mean
and variance
.
For this model the parameter
is fitted in
addition to the set of shared parameters.
Independent Gaussian Model (IG)
According to IG, is sampled independently
from
(Rausch & Zehetleitner, 2017).
is normally distributed with a mean of
and variance
of 1 (again as it would scale with
). The free parameter
represents the amount of information available for confidence judgment
relative to amount of evidence available for the discrimination decision and can
be smaller as well as greater than 1.
Independent Truncated Gaussian Model: HMetad-Version (ITGc)
According to the version of ITG consistent
with the HMetad-method (Fleming, 2017; see Rausch et al., 2023), is sampled independently
from
from a truncated Gaussian distribution with a location parameter
of
and a scale parameter of 1. The Gaussian distribution of
is truncated in a way that it is impossible to sample evidence that contradicts
the original decision: If
, the distribution is truncated to the
right of
. If
, the distribution is truncated to the left
of
. The additional parameter
represents metacognitive efficiency,
i.e., the amount of information available for confidence judgments relative to
amount of evidence available for discrimination decisions and can be smaller
as well as greater than 1.
Independent Truncated Gaussian Model: Meta-d'-Version (ITGcm)
According to the version of the ITG consistent
with the original meta-d' method (Maniscalco & Lau, 2012, 2014; see Rausch et al., 2023),
is sampled independently from
from a truncated Gaussian distribution with a location parameter
of
and a scale parameter
of 1. If
, the distribution is truncated to the right of
.
If
, the distribution is truncated to the left of
.
The additional parameter
represents metacognitive efficiency, i.e.,
the amount of information available for confidence judgments relative to
amount of evidence available for the discrimination decision and can be smaller
as well as greater than 1.
Logistic Noise Model (logN)
According to logN, the same sample
of sensory evidence is used to generate response and confidence, i.e.,
just as in SDT (Shekhar & Rahnev, 2021). However, according to logN, the confidence criteria
are not assumed to be constant, but instead they are affected by noise drawn from
a lognormal distribution. In each trial,
is given
by
. Likewise,
is given by
.
is drawn from a lognormal distribution with
the location parameter
and
scale parameter
.
is a free parameter designed to
quantify metacognitive ability. It is assumed that the criterion noise is perfectly
correlated across confidence criteria, ensuring that the confidence criteria
are always perfectly ordered. Because
, ...,
,
, ...,
change from trial to trial, they are not estimated
as free parameters. Instead, we estimate the means of the confidence criteria, i.e.,
,
as free parameters.
Logistic Weighted Evidence and Visibility model (logWEV)
logWEV is a combination of logN and WEV proposed by Shekhar and Rahnev (2023).
Conceptually, logWEV assumes that the observer combines evidence about decision-relevant features
of the stimulus with the strength of evidence about choice-irrelevant features (Rausch et al., 2018).
The model also assumes that noise affecting the confidence decision variable is lognormal
in accordance with Shekhar and Rahnev (2021).
According to logWEV, the confidence decision variable is equal to
.
is sampled from a lognormal distribution with a location parameter
of
and a scale parameter of
.
The parameter
quantifies the amount of unsystematic variability
contributing to confidence judgments but not to the discrimination judgments.
The parameter
represents the weight that is put on the choice-irrelevant
features in the confidence judgment.
and
are fitted in
addition to the set of shared parameters.
Value
Gives data frame with one row and columns for the fitted parameters of the
selected model as well as additional information about the fit
(negLogLik
(negative log-likelihood of the final set of parameters),
k
(number of parameters), N
(number of data rows),
AIC
(Akaike Information Criterion; Akaike, 1974),
BIC
(Bayes information criterion; Schwarz, 1978), and
AICc
(AIC corrected for small samples; Burnham & Anderson, 2002))
Author(s)
Sebastian Hellmann, sebastian.hellmann@ku.de
Manuel Rausch, manuel.rausch@hochschule-rhein-waal.de
References
Akaike, H. (1974). A New Look at the Statistical Model Identification. IEEE Transactions on Automatic Control, AC-19(6), 716–723.doi: 10.1007/978-1-4612-1694-0_16
Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information-theoretic approach. Springer.
Fleming, S. M. (2017). HMeta-d: Hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings. Neuroscience of Consciousness, 1, 1–14. doi: 10.1093/nc/nix007
Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. Wiley.
Maniscalco, B., & Lau, H. (2012). A signal detection theoretic method for estimating metacognitive sensitivity from confidence ratings. Consciousness and Cognition, 21(1), 422–430.
Maniscalco, B., & Lau, H. C. (2014). Signal Detection Theory Analysis of Type 1 and Type 2 Data: Meta-d’, Response- Specific Meta-d’, and the Unequal Variance SDT Model. In S. M. Fleming & C. D. Frith (Eds.), The Cognitive Neuroscience of Metacognition (pp. 25–66). Springer. doi: 10.1007/978-3-642-45190-4_3
Maniscalco, B., & Lau, H. (2016). The signal processing architecture underlying subjective reports of sensory awareness. Neuroscience of Consciousness, 1, 1–17. doi: 10.1093/nc/niw002
Rausch, M., Hellmann, S., & Zehetleitner, M. (2018). Confidence in masked orientation judgments is informed by both evidence and visibility. Attention, Perception, and Psychophysics, 80(1), 134–154. doi: 10.3758/s13414-017-1431-5
Rausch, M., Hellmann, S., & Zehetleitner, M. (2023). Measures of metacognitive efficiency across cognitive models of decision confidence. Psychological Methods. doi: 10.31234/osf.io/kdz34
Rausch, M., & Zehetleitner, M. (2017). Should metacognition be measured by logistic regression? Consciousness and Cognition, 49, 291–312. doi: 10.1016/j.concog.2017.02.007
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. doi: 10.1214/aos/1176344136
Shekhar, M., & Rahnev, D. (2021). The Nature of Metacognitive Inefficiency in Perceptual Decision Making. Psychological Review, 128(1), 45–70. doi: 10.1037/rev0000249
Shekhar, M., & Rahnev, D. (2023). How Do Humans Give Confidence? A Comprehensive Comparison of Process Models of Perceptual Metacognition. Journal of Experimental Psychology: General. doi:10.1037/xge0001524
Examples
# 1. Select one subject from the masked orientation discrimination experiment
data <- subset(MaskOri, participant == 1)
head(data)
# 2. Use fitting function
# Fitting takes some time to run:
FitFirstSbjWEV <- fitConf(data, model="WEV")