zCalibrate {pCalibrate}

Calibration of p-values from z-tests

Description

Transforms one- and two-sided p-values from z-tests to minimum Bayes factors.

Usage

zCalibrate(p=NULL, z=p2z(p), type="two.sided",  
           alternative="normal", transform="id")

Arguments

p

a vector of p-values, default is NULL

z

a vector of z-values, is calculated from p, if necessary.

type

either "one.sided" or "two.sided". Defaults to "two.sided". Specifies if the z-test (and the corresponding p-value) is one-sided or two-sided.

alternative

either "simple", "normal" or "local". Defaults to "normal". Specifies the alternative hypotheses for the mean to consider. See Details for more information.

transform

either "id", "log", "log2" or "log10". Defaults to "id". Specifies how to transform the minimum Bayes factor(s). "id" corresponds to no transformation. "log" refers to the natural logarithm, "log2" to the logarithm to the base 2 and "log10" to the logarithm to the base 10.

Details

For one-sided z-tests (type="one.sided"), alternative="simple" considers all simple point alternatives for the mean in the specified direction, alternative="normal" all shifted and possibly mirrored (if direction=less) half-normal distributions with mode at the null value and alternative="local" all monotonic density functions with mode at the null value, which are concentrated on the parameter space of the alternative.

For two-sided z-tests (type="two.sided"), alternative="simple" considers all two-point distributions on the mean which are symmetric with respect to the null value,
alternative="normal" all normal distributions centered around the null value and
alternative="local" all unimodal symmetric distributions with respect the null value (i.e. these distributions are non-increasing as a function of the distance to the null value).

Note that for alternative="normal" and alternative="local", the minimum Bayes factor is the same for type="one.sided" and type="two.sided".

For type="one.sided", alternative="simple" and for alternative="normal", the calibrations have closed-form expressions and were proposed in Edwards et al. (1963).
For type="two.sided", alternative="simple" and for alternative="local", the calibrations need to be computed by numerical optimization and were derived in Berger & Sellke (1987). Most of these calibrations are also described in Held & Ott (2018).

Note that if one considers the class of all alternatives, the resulting minimum Bayes factor is the same as for type="one.sided", alternative="simple".

One can show that if one considers the class of all alternatives which are symmetric around the null value, one obtains the same minimum Bayes factor as for the subclass of all symmetric two-point distributions (Berger & Sellke, 1987).

Value

A numeric vector of minimum Bayes factors for the specified p-values

Note

For small p-values, using the corresponding z-value as an argument instead of the p-value may lead to more accurate computations.

References

Berger, J. O. and Sellke, T. (1987). Testing a point null hypothesis: The irreconcilability of P values and evidence (with discussion). Journal of the American Statistical Association, 82, 112–139.

Edwards, W., Lindman, H. and Savage, L. J. (1963). Bayesian statistical inference for psychological research. Psychological Review, 70, 193–242.

Held, L. and Ott, M. (2018). On p-values and Bayes factors. Annual Review of Statistics and Its Application, 5, 393–419.

See Also

tCalibrate for sample-size adjusted calibrations of p-values from t-tests

Examples

  
# two-sided alternatives
zCalibrate(p=c(0.05, 0.01, 0.005), alternative="simple")
zCalibrate(p=c(0.05, 0.01, 0.005))
zCalibrate(p=c(0.05, 0.01, 0.005), alternative="local")
zCalibrate(z=c(2, 3, 4), alternative="local")
# one-sided alternatives
zCalibrate(p=c(0.05, 0.01, 0.005), type="one.sided", 
           alternative="simple")
zCalibrate(p=c(0.05, 0.01, 0.005), type="one.sided")
zCalibrate(p=c(0.05, 0.01, 0.005), type="one.sided", 
           alternative="local")

# plot the different calibrations as a function of the p-value
par(las=1)
p <- exp(seq(log(0.0001), log(0.3), by=0.01))
minBF <- matrix(NA, ncol=4, nrow=length(p))
minBF[ ,1] <- zCalibrate(p)
minBF[ ,2] <- zCalibrate(p, type="two.sided", alternative="local")
minBF[ ,3] <- zCalibrate(p, type="two.sided", alternative="simple")
minBF[ ,4] <- zCalibrate(p, type="one.sided", alternative="simple")
matplot(p, minBF, type="l", ylab="Minimum Bayes factor", log="xy", 
        xlab="z-test p-value", lty=1, lwd=2, col=c(1,3,2,4), axes=FALSE)
axis(1, at=c(0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3), 
     as.character(c(format(c(0.0001,0.0003), nsmall=4, digits=4, 
                  scientific=FALSE), 
                  c(0.001, 0.003, 0.01, 0.03, 0.1, 0.3))))
my.values <- c(3000, 1000, 300, 100, 30, 10, 3, 1)
my.at <- 1/my.values
my.ylegend <- c(paste("1/", my.values[-length(my.values)], sep=""), "1")
axis(2, at=my.at, my.ylegend)
box()
legend("bottomright", lty=1, lwd=2, 
        legend=c("normal", "unimodal symmetric", 
                "two-sided simple", "one-sided simple"), col=c(1,3,2,4))