FCalibrate {pCalibrate}

Calibration of two-sided p-values from the F-test in the linear model

Description

Transforms two-sided p-values from the F-test of overall significance in the linear model to sample-size adjusted lower bounds on the Bayes factor for the point null hypothesis against the alternative.

Usage

FCalibrate(p, n, d, alternative="chi.squared", intercept=TRUE, 
           transform="id")

Arguments

p

a vector of two-sided p-values

n

a scalar or a vector of positive integers. Specifies the sample size(s). May be a vector only if d is a scalar.

d

a scalar or a vector of positive integers. Specifies the dimension(s) of the vector(s) of regression coefficients, i.e. the number(s) of explanatory variables in the linear model(s). May be a vector only if n is a scalar.

alternative

either "simple" or "chi.squared". Defaults to "chi.squared".
Specifies the alternative hypotheses on the non-centrality parameter of the F-distribution to consider. "simple" only considers simple point alternative hypotheses. "chi.squared" assumes a scaled chi-squared distribution.

intercept

logical. If TRUE, the linear model contains an unknown intercept term, otherwise the intercept is fixed. Defaults to TRUE.

transform

either "id", "log", "log2" or "log10". Defaults to "id". Specifies how to transform the lower bound on the Bayes factor. "id" corresponds to no transformation. "log" refers to the natural logarithm, "log2" to the logarithm to the base 2 and "log10" to the logarithm to the base 10.

Details

Note that under the point null hypothesis that all regression coefficients are equal to zero, the F-statistic F (which is the (1-p)-quantile of the F-distribution with d and n-d-1 degrees of freedom) has a central F-distribution with d and n-d-1 degrees of freedom if the linear model contains an unknown intercept term (otherwise F has a central F-distribution with d and n-d degrees of freedom). Under a simple point alternative, F has a non-central F-distribution with d and n-d-1 degrees of freedom.

To obtain the lower bound on the Bayes factor for alternative="simple", the likelihood under the alternative is then maximized numerically with respect to the non-centrality parameter. That calibration is described in Held & Ott (2018), Section 3.1 (in the last two paragraphs).

For alternative="chi-squared", the calibration is proposed in Held & Ott (2016), Section 3 and also described in Held & Ott (2018), Section 3.2. The corresponding minimum Bayes factor has already been derived in Johnson (2005). As described there, assigning a scaled chi-sqaured distribution to the non-centrality parameter of the F-distribution corresponds to assigning a (multivariate) normal prior distribution centered around the null value to the vector of regression coefficients.

Value

A matrix containing the lower bounds on the Bayes factors as entries, for all combinations of p-value and sample size n or dimension d (whichever is multidimensional). The values for the k-th sample size or dimension (k-th entry in the vector n or d) and the different p-values are given in the k-th row.

Note

Computation may fail for alternative="simple" if the p-value p is extremely small and min{n, n-d} is also small. Warnings will be given in this case and the returned value is minBF=NaN.

References

Held , L. and Ott, M. (2016). How the maximal evidence of P -values against point null hypotheses depends on sample size. American Statistician, 70, 335–341

Held, L. and Ott, M. (2018). On p-values and Bayes factors. Annual Review of Statistics and Its Application, 5, 393–419.

Johnson, V. E. (2005). Bayes factors based on test statistics. Journal of the Royal Statistical Society, Series B 67:689–701.

See Also

tCalibrate

Examples

FCalibrate(p=c(0.05, 0.01, 0.005), n=20, d=2, alternative="simple")
# chi-squared alternatives
FCalibrate(p=c(0.05, 0.01, 0.005), n=20, d=2, intercept=FALSE)
FCalibrate(p=c(0.05, 0.01, 0.005), n=20, d=c(2, 5, 10))
FCalibrate(p=c(0.05, 0.01, 0.005), n=c(10, 20, 50), d=2)

# plot for chi-squared alternatives: d=2 and different sample sizes n
# note that the minimum Bayes factor decreases with decreasing sample 
# size
p <- exp(seq(log(0.0001), log(0.3), by=0.01))
n <- c(5, 10, 20)
minBF <- FCalibrate(p, n, d=2) 
# compare to the bound for large n
minTBF <- LRCalibrate(p, df=2)
par(las=1)
matplot(p, t(minBF), ylim=c(0.0003, 1), type="l", 
        xlab="two-sided F-test p-value", ylab="Minimum Bayes factor", 
        log="xy", lty=1, lwd=2, axes=FALSE, 
        main="Local normal alternatives")
lines(p, minTBF, col="gray", lty=2, lwd=2)
axis(1, at=c(0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3), 
     as.character(c(format(c(0.0001,0.0003), nsmall=4, digits=4, 
                           scientific=FALSE), 
                    c(0.001, 0.003, 0.01, 0.03, 0.1, 0.3))))
my.values <- c(3000, 1000, 300, 100, 30, 10, 3, 1)
my.at <- 1/my.values
my.ylegend <- c(paste("1/", my.values[-length(my.values)], sep=""), 
                "1")
axis(2, at=my.at, my.ylegend)
box()
legend("bottomright", 
       legend=rev(c("n=5", "n=10", "n=20", "n large")), 
       col=rev(c(1:3, "gray")), lty=c(2, rep(1, times=3)), lwd=2)