| pcal {pcal} | R Documentation |
Lower Bounds on the Posterior Probabilities of Point Null Hypotheses
Description
Calibrate p-values under a robust perspective so that they can be interpreted as either lower bounds on the posterior probabilities of point null hypotheses or as lower bounds on the probabilities of type I errors.
Usage
pcal(p, prior_prob = 0.5)
Arguments
p |
A numeric vector with values in the [0,1] interval. |
prior_prob |
A numeric vector with values in the [0,1] interval. If |
Details
Sellke et al. (2001) developed a calibration of p-values into lower bounds for the posterior probabilities of point null hypotheses or lower bounds for the probabilities of type I errors for the case when both the null and the alternative hypotheses have 0.5 prior probability. pcal generalizes the aforementioned calibration for prior probabilities other than 0.5.
pcal starts by transforming the values in p into lower bounds on Bayes factors using bcal and then uses bfactor_to_prob together with prior probabilities prior_prob to turn those Bayes factors into posterior probabilities. For each element of p, pcal returns an approximation of the smallest posterior probability of the null hypothesis that is found by changing the prior distribution of the parameter of interest (under the alternative hypothesis) over wide classes of distributions.
The prior_prob argument is optional and is set to 0.5 by default, implying prior equiprobability of hypotheses. prior_prob can only be of length equal to the length of p, in which case each prior probability in prior_prob is used in the calibration of the corresponding element of p, or of length 1, in which case it will be recycled (if length(p) > 1) and the same prior_prob value is used in the calibration of all the elements of p.
The output of pcal can also be interpreted as lower bounds on the probabilities of type I errors. Note that the output of this calibration has both Bayesian and Frequentist interpretations. Sellke et al. (2001) noted that a scenario in which they definitely recommend this calibration is when investigating fit to the null model with no explicit alternative in mind. Pericchi and Torres (2011) warn that despite the usefulness and appropriateness of this p-value calibration it does not depend on sample size, and hence the lower bounds obtained with large samples may be conservative.
Value
If length(p) > 1 then pcal returns a numeric vector with the same length as p, otherwise it returns a numeric vector with the same length as prior_prob.
References
Pericchi L, Torres D (2011).
“Quick anomaly detection by the Newcomb—Benford law, with applications to electoral processes data from the USA, Puerto Rico and Venezuela.”
Statistical Science, 26(4), 502–516.
Sellke T, Bayarri MJ, Berger JO (2001).
“Calibration of p values for testing precise null hypotheses.”
The American Statistician, 55(1), 62–71.
See Also
-
bcal for a p-value calibration that returns lower bounds on Bayes factors in favor of point null hypotheses.
Examples
# Calibration of a typical "threshold" p-value:
pcal(.05)
# Calibration of typical "threshold" p-values:
pcal(c(.1, .05, .01, .005, .001))
# Application: chi-squared goodness-of-fit test,
# lower bound on the posterior probability of the null hypothesis:
x <- matrix(c(12, 41, 25, 33), ncol = 2)
pcal(chisq.test(x)[["p.value"]])