tTestPower {EnvStats}  R Documentation 
Compute the power of a one or twosample ttest, given the sample size, scaled difference, and significance level.
tTestPower(n.or.n1, n2 = n.or.n1, delta.over.sigma = 0, alpha = 0.05,
sample.type = ifelse(!missing(n2), "two.sample", "one.sample"),
alternative = "two.sided", approx = FALSE)
n.or.n1 
numeric vector of sample sizes. When 
n2 
numeric vector of sample sizes for group 2. The default value is the value of

delta.over.sigma 
numeric vector specifying the ratio of the true difference The default value is 
alpha 
numeric vector of numbers between 0 and 1 indicating the Type I error level
associated with the hypothesis test. The default value is 
sample.type 
character string indicating whether to compute power based on a onesample or
twosample hypothesis test. When 
alternative 
character string indicating the kind of alternative hypothesis. The possible values are:

approx 
logical scalar indicating whether to compute the power based on an approximation to
the noncentral tdistribution. The default value is 
If the arguments n.or.n1
, n2
, delta.over.sigma
, and
alpha
are not all the same length, they are replicated to be the same
length as the length of the longest argument.
OneSample Case (sample.type="one.sample"
)
Let \underline{x} = x_1, x_2, \ldots, x_n
denote a vector of n
observations from a normal distribution with mean \mu
and standard deviation \sigma
, and consider the null hypothesis:
H_0: \mu = \mu_0 \;\;\;\;\;\; (1)
The three possible alternative hypotheses are the upper onesided alternative
(alternative="greater"
):
H_a: \mu > \mu_0 \;\;\;\;\;\; (2)
the lower onesided alternative (alternative="less"
)
H_a: \mu < \mu_0 \;\;\;\;\;\; (3)
and the twosided alternative (alternative="two.sided"
)
H_a: \mu \ne \mu_0 \;\;\;\;\;\; (4)
The test of the null hypothesis (1) versus any of the three alternatives (2)(4) is based on the Student tstatistic:
t = \frac{\bar{x}  \mu_0}{s/\sqrt{n}} \;\;\;\;\;\; (5)
where
\bar{x} = \frac{1}{n} \sum_{i=1}^n x_i \;\;\;\;\;\; (6)
s^2 = \frac{1}{n1} \sum_{i=1}^n (x_i  \bar{x})^2 \;\;\;\;\;\; (7)
Under the null hypothesis (1), the tstatistic in (5) follows a
Student's tdistribution with n1
degrees of freedom
(Zar, 2010, Chapter 7; Johnson et al., 1995, pp.362363).
The formula for the power of the test depends on which alternative is being tested.
The two subsections below describe exact and approximate formulas for the power of
the onesample ttest. Note that none of the equations for the power of the ttest
requires knowledge of the values \delta
(Equation (12) below) or \sigma
(the population standard deviation), only the ratio \delta/\sigma
. The
argument delta.over.sigma
is this ratio, and it is referred to as the
“scaled difference”.
Exact Power Calculations (approx=FALSE
)
This subsection describes the exact formulas for the power of the onesample
ttest.
Upper onesided alternative (alternative="greater"
)
The standard Student's ttest rejects the null hypothesis (1) in favor of the
upper alternative hypothesis (2) at level\alpha
if
t \ge t_{\nu}(1  \alpha) \;\;\;\;\;\; (8)
where
\nu = n  1 \;\;\;\;\;\; (9)
and t_{\nu}(p)
denotes the p
'th quantile of Student's tdistribution
with \nu
degrees of freedom (Zar, 2010; Berthouex and Brown, 2002).
The power of this test, denoted by 1\beta
, where \beta
denotes the
probability of a Type II error, is given by:
1  \beta = Pr[t_{\nu, \Delta} \ge t_{\nu}(1  \alpha)] = 1  G[t_{\nu}(1  \alpha), \nu, \Delta] \;\;\;\;\;\; (10)
where
\Delta = \sqrt{n} (\frac{\delta}{\sigma}) \;\;\;\;\;\; (11)
\delta = \mu  \mu_0 \;\;\;\;\;\ (12)
and t_{\nu, \Delta}
denotes a
noncentral Student's trandom variable with
\nu
degrees of freedom and noncentrality parameter \Delta
, and
G(x, \nu, \Delta)
denotes the cumulative distribution function of this
random variable evaluated at x
(Johnson et al., 1995, pp.508510).
Lower onesided alternative (alternative="less"
)
The standard Student's ttest rejects the null hypothesis (1) in favor of the
lower alternative hypothesis (3) at level\alpha
if
t \le t_{\nu}(\alpha) \;\;\;\;\;\; (13)
and the power of this test is given by:
1  \beta = Pr[t_{\nu, \Delta} \le t_{\nu}(\alpha)] = G[t_{\nu}(\alpha), \nu, \Delta] \;\;\;\;\;\; (14)
Twosided alternative (alternative="two.sided"
)
The standard Student's ttest rejects the null hypothesis (1) in favor of the
twosided alternative hypothesis (4) at level\alpha
if
t \ge t_{\nu}(1  \alpha/2) \;\;\;\;\;\; (15)
and the power of this test is given by:
1  \beta = Pr[t_{\nu, \Delta} \le t_{\nu}(\alpha/2)] + Pr[t_{\nu, \Delta} \ge t_{\nu}(1  \alpha/2)]
= G[t_{\nu}(\alpha/2), \nu, \Delta] + 1  G[t_{\nu}(1  \alpha/2), \nu, \Delta] \;\;\;\;\;\; (16)
The power of the ttest given in Equation (16) can also be expressed in terms of the
cumulative distribution function of the noncentral Fdistribution
as follows. Let F_{\nu_1, \nu_2, \Delta}
denote a
noncentral F random variable with \nu_1
and
\nu_2
degrees of freedom and noncentrality parameter \Delta
, and let
H(x, \nu_1, \nu_2, \Delta)
denote the cumulative distribution function of this
random variable evaluated at x
. Also, let F_{\nu_1, \nu_2}(p)
denote
the p
'th quantile of the central Fdistribution with \nu_1
and
\nu_2
degrees of freedom. It can be shown that
(t_{\nu, \Delta})^2 \cong F_{1, \nu, \Delta^2} \;\;\;\;\;\; (17)
where \cong
denotes “equal in distribution”. Thus, it follows that
[t_{\nu}(1  \alpha/2)]^2 = F_{1, \nu}(1  \alpha) \;\;\;\;\;\; (18)
so the formula for the power of the ttest given in Equation (16) can also be written as:
1  \beta = Pr\{(t_{\nu, \Delta})^2 \ge [t_{\nu}(1  \alpha/2)]^2\}
= Pr[F_{1, \nu, \Delta^2} \ge F_{1, \nu}(1  \alpha)] = 1  H[F_{1, \nu}(1\alpha), 1, \nu, \Delta^2] \;\;\;\;\;\; (19)
Approximate Power Calculations (approx=TRUE
)
Zar (2010, pp.115–118) presents an approximation to the power for the ttest
given in Equations (10), (14), and (16) above. His approximation to the power
can be derived by using the approximation
\sqrt{n} (\frac{\delta}{s}) \approx \sqrt{n} (\frac{\delta}{\sigma}) = \Delta \;\;\;\;\;\; (20)
where \approx
denotes “approximately equal to”. Zar's approximation
can be summarized in terms of the cumulative distribution function of the
noncentral tdistribution as follows:
G(x, \nu, \Delta) \approx G(x  \Delta, \nu, 0) = G(x  \Delta, \nu) \;\;\;\;\;\; (21)
where G(x, \nu)
denotes the cumulative distribution function of the
central Student's tdistribution with \nu
degrees of freedom evaluated at
x
.
The following three subsections explicitly derive the approximation to the power of
the ttest for each of the three alternative hypotheses.
Upper onesided alternative (alternative="greater"
)
The power for the upper onesided alternative (2) given in Equation (10) can be
approximated as:
1  \beta = Pr[t \ge t_{\nu}(1  \alpha)]
= Pr[\frac{\bar{x}  \mu}{s/\sqrt{n}} \ge t_{\nu}(1  \alpha)  \sqrt{n}\frac{\delta}{s}]
\approx Pr[t_{\nu} \ge t_{\nu}(1  \alpha)  \Delta]
= 1  Pr[t_{\nu} \le t_{\nu}(1  \alpha)  \Delta]
= 1  G[t_{\nu}(1\alpha)  \Delta, \nu] \;\;\;\;\;\; (22)
where t_{\nu}
denotes a central Student's trandom variable with \nu
degrees of freedom.
Lower onesided alternative (alternative="less"
)
The power for the lower onesided alternative (3) given in Equation (14) can be
approximated as:
1  \beta = Pr[t \le t_{\nu}(\alpha)]
= Pr[\frac{\bar{x}  \mu}{s/\sqrt{n}} \le t_{\nu}(\alpha)  \sqrt{n}\frac{\delta}{s}]
\approx Pr[t_{\nu} \le t_{\nu}(\alpha)  \Delta]
= G[t_{\nu}(\alpha)  \Delta, \nu] \;\;\;\;\;\; (23)
Twosided alternative (alternative="two.sided"
)
The power for the twosided alternative (4) given in Equation (16) can be
approximated as:
1  \beta = Pr[t \le t_{\nu}(\alpha/2)] + Pr[t \ge t_{\nu}(1  \alpha/2)]
= Pr[\frac{\bar{x}  \mu}{s/\sqrt{n}} \le t_{\nu}(\alpha/2)  \sqrt{n}\frac{\delta}{s}] + Pr[\frac{\bar{x}  \mu}{s/\sqrt{n}} \ge t_{\nu}(1  \alpha)  \sqrt{n}\frac{\delta}{s}]
\approx Pr[t_{\nu} \le t_{\nu}(\alpha/2)  \Delta] + Pr[t_{\nu} \ge t_{\nu}(1  \alpha/2)  \Delta]
= G[t_{\nu}(\alpha/2)  \Delta, \nu] + 1  G[t_{\nu}(1\alpha/2)  \Delta, \nu] \;\;\;\;\;\; (24)
TwoSample Case (sample.type="two.sample"
)
Let \underline{x}_1 = x_{11}, x_{12}, \ldots, x_{1n_1}
denote a vector of
n_1
observations from a normal distribution with mean
\mu_1
and standard deviation \sigma
, and let
\underline{x}_2 = x_{21}, x_{22}, \ldots, x_{2n_2}
denote a vector of
n_2
observations from a normal distribution with mean \mu_2
and
standard deviation \sigma
, and consider the null hypothesis:
H_0: \mu_1 = \mu_2 \;\;\;\;\;\; (25)
The three possible alternative hypotheses are the upper onesided alternative
(alternative="greater"
):
H_a: \mu_1 > \mu_2 \;\;\;\;\;\; (26)
the lower onesided alternative (alternative="less"
)
H_a: \mu_1 < \mu_2 \;\;\;\;\;\; (27)
and the twosided alternative (alternative="two.sided"
)
H_a: \mu_1 \ne \mu_2 \;\;\;\;\;\; (28)
The test of the null hypothesis (25) versus any of the three alternatives (26)(28) is based on the Student tstatistic:
t = \frac{\bar{x}_1  \bar{x}_2}{s_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}} \;\;\;\;\;\; (29)
where
\bar{x}_1 = \frac{1}{n_1} \sum_{i=1}^{n_1} x_{1i} \;\;\;\;\;\; (30)
\bar{x}_2 = \frac{1}{n_2} \sum_{i=1}^{n_2} x_{2i} \;\;\;\;\;\; (31)
s_p^2 = \frac{(n_1  1) s_1^2 + (n_2  1) s_2^2}{n_1 + n_2  2} \;\;\;\;\;\; (32)
s_1^2 = \frac{1}{n_1  1} \sum_{i=1}^{n_1} (x_{1i}  \bar{x}_1)^2 \;\;\;\;\;\; (33)
s_2^2 = \frac{1}{n_2  1} \sum_{i=1}^{n_2} (x_{2i}  \bar{x}_2)^2 \;\;\;\;\;\; (34)
Under the null hypothesis (25), the tstatistic in (29) follows a
Student's tdistribution with n_1 + n_2  2
degrees of
freedom (Zar, 2010, Chapter 8; Johnson et al., 1995, pp.508–510,
Helsel and Hirsch, 1992, pp.124–128).
The formulas for the power of the twosample ttest are precisely the same as those for the onesample case, with the following modifications:
\nu = n_1 + n_2  2 \;\;\;\;\;\; (35)
\Delta = \sqrt{\frac{n_1 n_2}{n_1 + n_2}} (\frac{\delta}{\sigma}) \;\;\;\;\;\; (36)
\delta = \mu_1  \mu_2 \;\;\;\;\;\; (37)
Note that none of the equations for the power of the ttest requires knowledge of
the values \delta
or \sigma
(the population standard deviation for
both populations), only the ratio \delta/\sigma
. The argument
delta.over.sigma
is this ratio, and it is referred to as the
“scaled difference”.
a numeric vector powers.
The normal distribution and lognormal distribution are probably the two most frequently used distributions to model environmental data. Often, you need to determine whether a population mean is significantly different from a specified standard (e.g., an MCL or ACL, USEPA, 1989b, Section 6), or whether two different means are significantly different from each other (e.g., USEPA 2009, Chapter 16). In this case, assuming normally distributed data, you can perform the Student's ttest.
In the course of designing a sampling program, an environmental scientist may wish
to determine the relationship between sample size, significance level, power, and
scaled difference if one of the objectives of the sampling program is to determine
whether a mean differs from a specified level or two means differ from each other.
The functions tTestPower
, tTestN
,
tTestScaledMdd
, and plotTTestDesign
can be used to
investigate these relationships for the case of normallydistributed observations.
Steven P. Millard (EnvStats@ProbStatInfo.com)
Berthouex, P.M., and L.C. Brown. (2002). Statistics for Environmental Engineers. Second Edition. Lewis Publishers, Boca Raton, FL.
Helsel, D.R., and R.M. Hirsch. (1992). Statistical Methods in Water Resources Research. Elsevier, New York, NY, Chapter 7.
Johnson, N. L., S. Kotz, and N. Balakrishnan. (1995). Continuous Univariate Distributions, Volume 2. Second Edition. John Wiley and Sons, New York, Chapters 28, 31
Millard, S.P., and N.K. Neerchal. (2001). Environmental Statistics with SPLUS. CRC Press, Boca Raton, FL.
USEPA. (1989b). Statistical Analysis of GroundWater Monitoring Data at RCRA Facilities, Interim Final Guidance. EPA/530SW89026. Office of Solid Waste, U.S. Environmental Protection Agency, Washington, D.C.
USEPA. (2009). Statistical Analysis of Groundwater Monitoring Data at RCRA Facilities, Unified Guidance. EPA 530/R09007, March 2009. Office of Resource Conservation and Recovery Program Implementation and Information Division. U.S. Environmental Protection Agency, Washington, D.C.
Zar, J.H. (2010). Biostatistical Analysis. Fifth Edition. PrenticeHall, Upper Saddle River, NJ.
tTestN
, tTestScaledMdd
, tTestAlpha
,
plotTTestDesign
, Normal,
t.test
, Hypothesis Tests.
# Look at how the power of the onesample ttest increases with
# increasing sample size:
seq(5, 30, by = 5)
#[1] 5 10 15 20 25 30
power < tTestPower(n.or.n1 = seq(5, 30, by = 5), delta.over.sigma = 0.5)
round(power, 2)
#[1] 0.14 0.29 0.44 0.56 0.67 0.75
#
# Repeat the last example, but use the approximation.
# Note how the approximation underestimates the power
# for the smaller sample sizes.
#
power < tTestPower(n.or.n1 = seq(5, 30, by = 5), delta.over.sigma = 0.5,
approx = TRUE)
round(power, 2)
#[1] 0.10 0.26 0.42 0.56 0.67 0.75
#
# Look at how the power of the twosample ttest increases with increasing
# scaled difference:
seq(0.5, 2, by = 0.5)
#[1] 0.5 1.0 1.5 2.0
power < tTestPower(10, sample.type = "two.sample",
delta.over.sigma = seq(0.5, 2, by = 0.5))
round(power, 2)
#[1] 0.19 0.56 0.89 0.99
#
# Look at how the power of the twosample ttest increases with increasing values
# of Type I error:
power < tTestPower(20, sample.type = "two.sample", delta.over.sigma = 0.5,
alpha = c(0.001, 0.01, 0.05, 0.1))
round(power, 2)
#[1] 0.03 0.14 0.34 0.46
#==========
# Modifying the example on pages 214 to 215 of USEPA (2009), determine how
# adding another four months of observations to increase the sample size from
# 4 to 8 for any one particular compliance well will affect the power of a
# onesample ttest that compares the mean for the well with the MCL of
# 7 ppb. Use alpha = 0.01, assume an upper onesided alternative
# (i.e., compliance well mean larger than 7 ppb), and assume a scaled
# difference of 2. (The data are stored in EPA.09.Ex.21.1.aldicarb.df.)
# Note that the power changes from 49% to 98% by increasing the sample size
# from 4 to 8.
tTestPower(n.or.n1 = c(4, 8), delta.over.sigma = 2, alpha = 0.01,
sample.type = "one.sample", alternative = "greater")
#[1] 0.4865800 0.9835401
#==========
# Clean up
#
rm(power)