chenTTest {EnvStats}  R Documentation 
Chen's Modified OneSided ttest for Skewed Distributions
Description
For a skewed distribution, estimate the mean, standard deviation, and skew; test the null hypothesis that the mean is equal to a userspecified value vs. a onesided alternative; and create a onesided confidence interval for the mean.
Usage
chenTTest(x, y = NULL, alternative = "greater", mu = 0, paired = !is.null(y),
conf.level = 0.95, ci.method = "z")
Arguments
x 
numeric vector of observations. Missing ( 
y 
optional numeric vector of observations that are paired with the observations in

alternative 
character string indicating the kind of alternative hypothesis. The possible values
are 
mu 
numeric scalar indicating the hypothesized value of the mean. The default value is

paired 
character string indicating whether to perform a paired or onesample ttest. The
possible values are 
conf.level 
numeric scalar between 0 and 1 indicating the confidence level associated with the
confidence interval for the population mean. The default value is 
ci.method 
character string indicating which critical value to use to construct the confidence
interval for the mean. The possible values are 
Details
OneSample Case (paired=FALSE
)
Let \underline{x} = (x_1, x_2, \ldots, x_n)
be a vector of n
independent
and identically distributed (i.i.d.) observations from some distribution with mean
\mu
and standard deviation \sigma
.
Background: The Conventional Student's tTest
Assume that the n
observations come from a normal (Gaussian) distribution, and
consider the test of the null hypothesis:
H_0: \mu = \mu_0 \;\;\;\;\;\; (1)
The three possible alternative hypotheses are the upper onesided alternative
(alternative="greater"
):
H_a: \mu > \mu_0 \;\;\;\;\;\; (2)
the lower onesided alternative (alternative="less"
):
H_a: \mu < \mu_0 \;\;\;\;\;\; (3)
and the twosided alternative:
H_a: \mu \ne \mu_0 \;\;\;\;\;\; (4)
The test of the null hypothesis (1) versus any of the three alternatives (2)(4) is usually based on the Student tstatistic:
t = \frac{\bar{x}  \mu_0}{s/\sqrt{n}} \;\;\;\;\;\; (5)
where
\bar{x} = \frac{1}{n}\sum_{i=1}^n x_i \;\;\;\;\;\; (6)
s^2 = \frac{1}{n1} \sum_{i=1}^n (x_i  \bar{x})^2 \;\;\;\;\;\; (7)
(see the R help file for t.test
). Under the null hypothesis (1),
the tstatistic in (5) follows a Student's tdistribution with
n1
degrees of freedom (Zar, 2010, p.99; Johnson et al., 1995, pp.362363).
The tstatistic is fairly robust to departures from normality in terms of
maintaining Type I error and power, provided that the sample size is sufficiently
large.
Chen's Modified tTest for Skewed Distributions
In the case when the underlying distribution of the n
observations is
positively skewed and the sample size is small, the sampling distribution of the
tstatistic under the null hypothesis (1) does not follow a Student's tdistribution,
but is instead negatively skewed. For the test against the upper alternative in (2)
above, this leads to a Type I error smaller than the one assumed and a loss of power
(Chen, 1995b, p.767).
Similarly, in the case when the underlying distribution of the n
observations
is negatively skewed and the sample size is small, the sampling distribution of the
tstatistic is positively skewed. For the test against the lower alternative in (3)
above, this also leads to a Type I error smaller than the one assumed and a loss of
power.
In order to overcome these problems, Chen (1995b) proposed the following modified tstatistic that takes into account the skew of the underlying distribution:
t_2 = t + a(1 + 2t^2) + 4a^2(t + 2t^3) \;\;\;\;\;\; (8)
where
a = \frac{\sqrt{\hat{\beta}_1}}{6n} \;\;\;\;\;\; (9)
\hat{\beta}_1 = \frac{\hat{\mu}_3}{\hat{\sigma}^3} \;\;\;\;\;\; (10)
\hat{\mu}_3 = \frac{n}{(n1)(n2)} \sum_{i=1}^n (x_i  \bar{x})^3 \;\;\;\;\;\; (11)
\hat{\sigma}^3 = s^3 = [\frac{1}{n1} \sum_{i=1}^n (x_i  \bar{x})^2]^{3/2} \;\;\;\;\;\; (12)
Note that the quantity \sqrt{\hat{\beta}_1}
in (9) is an estimate of
the skew of the underlying distribution and is based on unbiased estimators of
central moments (see the help file for skewness
).
For a positivelyskewed distribution, Chen's modified ttest rejects the null hypothesis (1) in favor of the upper onesided alternative (2) if the tstatistic in (8) is too large. For a negativelyskewed distribution, Chen's modified ttest rejects the null hypothesis (1) in favor of the lower onesided alternative (3) if the tstatistic in (8) is too small.
Chen's modified ttest is not applicable to testing the twosided alternative
(4). It should also not be used to test the upper onesided alternative (2)
based on negativelyskewed data, nor should it be used to test the lower onesided
alternative (3) based on positivelyskewed data.
Determination of Critical Values and pValues
Chen (1995b) performed a simulation study in which the modified tstatistic in (8)
was compared to a critical value based on the normal distribution (zvalue),
a critical value based on Student's tdistribution (tvalue), and the average of the
critical zvalue and tvalue. Based on the simulation study, Chen (1995b) suggests
using either the zvalue or average of the zvalue and tvalue when n
(the sample size) is small (e.g., n \le 10
) or \alpha
(the Type I error)
is small (e.g. \alpha \le 0.01
), and using either the tvalue or the average
of the zvalue and tvalue when n \ge 20
or \alpha \ge 0.05
.
The function chenTTest
returns three different pvalues: one based on the
normal distribution, one based on Student's tdistribution, and one based on the
average of these two pvalues. This last pvalue should roughly correspond to a
pvalue based on the distribution of the average of a normal and Student's t
random variable.
Computing Confidence Intervals
The function chenTTest
computes a onesided confidence interval for the true
mean \mu
based on finding all possible values of \mu
for which the null
hypothesis (1) will not be rejected, with the confidence level determined by the
argument conf.level
. The argument ci.method
determines which pvalue
is used in the algorithm to determine the bounds on \mu
. When
ci.method="z"
, the pvalue is based on the normal distribution, when
ci.method="t"
, the pvalue is based on Student's tdistribution, and when
ci.method="Avg. of z and t"
the pvalue is based on the average of the
pvalues based on the normal and Student's tdistribution.
PairedSample Case (paired=TRUE
)
When the argument paired=TRUE
, the arguments x
and y
are assumed
to have the same length, and the n
differences
d_i = x_i  y_i, \;\; i = 1, 2, \ldots, n
are assumed to be i.i.d. observations from some distribution with mean \mu
and standard deviation \sigma
. Chen's modified ttest can then be applied
to the differences.
Value
a list of class "htest"
containing the results of the hypothesis test. See
the help file for htest.object
for details.
Note
The presentation of Chen's (1995b) method in USEPA (2002d) and Singh et al. (2010b, p. 52) is incorrect for two reasons: it is based on an intermediate formula instead of the actual statistic that Chen proposes, and it uses the intermediate formula to compute an upper confidence limit for the mean when the sample data are positively skewed. As explained above, for the case of positively skewed data, Chen's method is appropriate to test the upper onesided alternative hypothesis that the population mean is greater than some specified value, and a onesided upper alternative corresponds to creating a onesided lower confidence limit, not an upper confidence limit (see, for example, Millard and Neerchal, 2001, p. 371).
A frequent question in environmental statistics is “Is the concentration of chemical X greater than Y units?” For example, in groundwater assessment (compliance) monitoring at hazardous and solid waste sites, the concentration of a chemical in the groundwater at a downgradient may be compared to a groundwater protection standard (GWPS). If the concentration is “above” the GWPS, then the site enters corrective action monitoring. As another example, soil screening at a Superfund site involves comparing the concentration of a chemical in the soil with a predetermined soil screening level (SSL). If the concentration is “above” the SSL, then further investigation and possible remedial action is required. Determining what it means for the chemical concentration to be “above” a GWPS or an SSL is a policy decision: the average of the distribution of the chemical concentration must be above the GWPS or SSL, or the median must be above the GWPS or SSL, or the 95'th percentile must be above the GWPS or SSL, or something else. Often, the first interpretation is used.
The regulatory guidance document Soil Screening Guidance: Technical Background Document (USEPA, 1996c, Part 4) recommends using Chen's ttest as one possible method to compare chemical concentrations in soil samples to a soil screening level (SSL). The document notes that the distribution of chemical concentrations will almost always be positivelyskewed, but not necessarily fit a lognormal distribution well (USEPA, 1996c, pp.107, 117119). It also notes that using a confidence interval based on Land's (1971) method is extremely sensitive to the assumption of a lognormal distribution, while Chen's test is robust with respect to maintaining Type I and Type II errors for a variety of positivelyskewed distributions (USEPA, 1996c, pp.99, 117119, 123125).
Hypothesis tests you can use to perform tests of location include: Student's ttest, Fisher's randomization test, the Wilcoxon signed rank test, Chen's modified ttest, the sign test, and a test based on a bootstrap confidence interval. For a discussion comparing the performance of these tests, see Millard and Neerchal (2001, pp.408–409).
Author(s)
Steven P. Millard (EnvStats@ProbStatInfo.com)
References
Chen, L. (1995b). Testing the Mean of Skewed Distributions. Journal of the American Statistical Association 90(430), 767–772.
Johnson, N. L., S. Kotz, and N. Balakrishnan. (1995). Continuous Univariate Distributions, Volume 2. Second Edition. John Wiley and Sons, New York, Chapters 28, 31.
Land, C.E. (1971). Confidence Intervals for Linear Functions of the Normal Mean and Variance. The Annals of Mathematical Statistics 42(4), 1187–1205.
Millard, S.P., and N.K. Neerchal. (2001). Environmental Statistics with SPLUS. CRC Press, Boca Raton, FL, pp.402–404.
Singh, A., N. Armbya, and A. Singh. (2010b). ProUCL Version 4.1.00 Technical Guide (Draft). EPA/600/R07/041, May 2010. Office of Research and Development, U.S. Environmental Protection Agency, Washington, D.C.
USEPA. (1996c). Soil Screening Guidance: Technical Background Document. EPA/540/R95/128, PB96963502. Office of Emergency and Remedial Response, U.S. Environmental Protection Agency, Washington, D.C., May, 1996.
USEPA. (2002d). Estimation of the Exposure Point Concentration Term Using a Gamma Distribution. EPA/600/R02/084. October 2002. Technology Support Center for Monitoring and Site Characterization, Office of Research and Development, Office of Solid Waste and Emergency Response, U.S. Environmental Protection Agency, Washington, D.C.
Zar, J.H. (2010). Biostatistical Analysis. Fifth Edition. PrenticeHall, Upper Saddle River, NJ.
See Also
Examples
# The guidance document "Calculating Upper Confidence Limits for
# Exposure Point Concentrations at Hazardous Waste Sites"
# (USEPA, 2002d, Exhibit 9, p. 16) contains an example of 60 observations
# from an exposure unit. Here we will use Chen's modified ttest to test
# the null hypothesis that the average concentration is less than 30 mg/L
# versus the alternative that it is greater than 30 mg/L.
# In EnvStats these data are stored in the vector EPA.02d.Ex.9.mg.per.L.vec.
sort(EPA.02d.Ex.9.mg.per.L.vec)
# [1] 16 17 17 17 18 18 20 20 20 21 21 21 21 21 21 22
#[17] 22 22 23 23 23 23 24 24 24 25 25 25 25 25 25 26
#[33] 26 26 26 27 27 28 28 28 28 29 29 30 30 31 32 32
#[49] 32 33 33 35 35 97 98 105 107 111 117 119
dev.new()
hist(EPA.02d.Ex.9.mg.per.L.vec, col = "cyan", xlab = "Concentration (mg/L)")
# The ShapiroWilk goodnessoffit test rejects the null hypothesis of a
# normal, lognormal, and gamma distribution:
gofTest(EPA.02d.Ex.9.mg.per.L.vec)$p.value
#[1] 2.496781e12
gofTest(EPA.02d.Ex.9.mg.per.L.vec, dist = "lnorm")$p.value
#[1] 3.349035e09
gofTest(EPA.02d.Ex.9.mg.per.L.vec, dist = "gamma")$p.value
#[1] 1.564341e10
# Use Chen's modified ttest to test the null hypothesis that
# the average concentration is less than 30 mg/L versus the
# alternative that it is greater than 30 mg/L.
chenTTest(EPA.02d.Ex.9.mg.per.L.vec, mu = 30)
#Results of Hypothesis Test
#
#
#Null Hypothesis: mean = 30
#
#Alternative Hypothesis: True mean is greater than 30
#
#Test Name: Onesample tTest
# Modified for
# PositivelySkewed Distributions
# (Chen, 1995)
#
#Estimated Parameter(s): mean = 34.566667
# sd = 27.330598
# skew = 2.365778
#
#Data: EPA.02d.Ex.9.mg.per.L.vec
#
#Sample Size: 60
#
#Test Statistic: t = 1.574075
#
#Test Statistic Parameter: df = 59
#
#Pvalues: z = 0.05773508
# t = 0.06040889
# Avg. of z and t = 0.05907199
#
#Confidence Interval for: mean
#
#Confidence Interval Method: Based on z
#
#Confidence Interval Type: Lower
#
#Confidence Level: 95%
#
#Confidence Interval: LCL = 29.82
# UCL = Inf
# The estimated mean, standard deviation, and skew are 35, 27, and 2.4,
# respectively. The pvalue is 0.06, and the lower 95% confidence interval
# is [29.8, Inf). Depending on what you use for your Type I error rate, you
# may or may not want to reject the null hypothesis.