RPT {RATest}

Robust Permutation Test

Description

This function considers the k-sample problem of comparing general parameters, such as means, medians, or parameters that depend on the joint distribution using permutation tests. Under weak assumptions for comparing estimator, the permutation tests implemented here provide a general test procedure whereby the asymptotic validity of the permutation test holds while retaining the exact rejection probability \alpha in finite samples when the underlying distributions are identical. Here we will consider three test for the 2 sample case, but the function works for k-samples.

Difference of means: Here, the null hypothesis is of the form H_0: \mu(P)-\mu(Q)=0, and the corresponding test statistic is given by

T_{m,n}=\frac{N^{1/2}(\bar{X}_m-\bar{Y}_n)}{\sqrt{\frac{N}{m}\sigma^2_m(X_1,\dots,X_m)+ \frac{N}{n}\sigma^2_n(Y_1,\dots,Y_n)}}

where \bar{X}_m and \bar{Y}_n are the sample means from population P and population Q, respectively, and \sigma^2_m(X_1,\dots,X_m) is a consistent estimator of \sigma^2(P) when X_1,\dots,X_m are i.i.d. from P. Assume consistency also under Q.

Difference of medians: Let F and G be the CDFs corresponding to P and Q, and denote \theta(F) the median of F i.e. \theta(F)=\inf\{x:F(x)\ge1/2\}. Assume that F is continuously differentiable at \theta(P) with derivative F' (and the same with F replaced by G). Here, the null hypothesis is of the form H_0: \theta(P)-\theta(Q)=0, and the corresponding test statistic is given by

T_{m,n}=\frac{N^{1/2}\left(\theta(\hat{P}_m)-\theta(\hat{Q})\right)}{\hat{\upsilon}_{m,n}}

where \hat{\upsilon}_{m,n} is a consistent estimator of \upsilon(P,Q):

\upsilon(P,Q)=\frac{1}{\lambda}\frac{1}{4(F'(\theta))^2}+\frac{1}{1-\lambda}\frac{1}{4(G'(\theta))^2}

Choices of \hat{\upsilon}_{m,n} may include the kernel estimator of Devroye and Wagner (1980), the bootstrap estimator of Efron (1992), or the smoothed bootstrap Hall et al. (1989) to list a few. For further details, see Chung and Romano (2013). Current implementation uses the bootstrap estimator of Efron (1992)

Difference of variances: Here, the null hypothesis is of the form H_0: \sigma^2(P)-\sigma^2(Q)=0, and the corresponding test statistic is given by

T_{m,n}=\frac{N^{1/2}(\hat{\sigma}_m^2(X_1,\dots,X_,)-\hat{\sigma}_n^2(Y_1,\dots,Y_n))}{\sqrt{\frac{N}{m}(\hat{\mu}_{4,x}-\frac{(m-3)}{(m-1)}(\hat{\sigma}_m^2)^2)+\frac{N}{n}(\hat{\mu}_{4,y}-\frac{(n-3)}{(n-1)}(\hat{\sigma}_y^2)^2)}}

where \hat{\mu}_{4,m} the sample analog of E(X-\mu)^4 based on an i.i.d. sample X_1,\dots,X_m from P. Similarly for \hat{\mu}_{4,n}.

We could also have the case when the parameter of interest is a function of the joint distribution. The examples considered here are

Lehmann (1951) two-sample U statistics: Consider testing H_0: P=Q, or the more general hypothesis that P and Q only differ in location against the alternative that the Y's are more spread out than the X's. The null hypothesis is of the form

H_0: P(\vert Y-Y'\vert>\vert X-X'\vert)=1/2

.

Two-sample Wilcoxon statistic, where the null hypothesis is of the form

H_0: P(X\le Y)=1/2

.

Two-sample Wilcoxon statistic without continuity assumption. In this case, the null hypothesis is of the form

H_0: P(X\le Y)=P(Y\le X)

.

Hollander (1967) two-sample U statistics. The null hypothesis is of the form

H_0: P(X+X'<Y+Y')=1/2

.

Usage

RPT(
  formula,
  data,
  test = "means",
  n.perm = 499,
  na.action,
  wilcoxon.option = "continuity"
)

Arguments

formula

a formula object, with the response on the left of a ~ operator, and the groups on the right.

data

a data.frame in which to interpret the variables named in the formula. If this is missing, then the variables in the formula should be on the search list.

test

test to be perfomed. Multiple options are available, depending on the nature of the testing problem. In general, we have two types of problem. First, when the researcher is interested in comparing parameters. In this case, "means" will perform a Difference of Means, "medians" a Difference of Medians, "variances" a Difference of Variances. This case allows for 2 or more population comparisons. For the test of difference of medians the Efron (1992) bootstrap estimator is used to estimate the variances (for further details, see Chung and Romano (2013)). Second, when the parameter of interest is a function of the joint distribution. In this case, "lehmann.2S.test" will perform Lehmann (1951) two-sample U statistics, "wilcoxon.2s.test" the two-sample Wilcoxon test (with or without continuity assumption), and "hollander.2S.test" Hollander (1967) two sample U statistics. In this case, only 2 sample comparisons are permitted.

n.perm

Numeric. Number of permutations needed for the stochastic approximation of the p-values. See remark 3.2 in Canay and Kamat (2017). The default is n.perm=499.

na.action

a function to filter missing data. This is applied to the model.frame . The default is na.omit, which deletes observations that contain one or more missing values.

wilcoxon.option

Continuity assumption for Wilcoxon test" with continuity ("continuity") or without ("discontinuity"). The default is "continuity"

Value

An object of class "RPT" is a list containing at least the following components:

description

Type of test, can be Difference of Means, Medians, or Variances.

n_populations

Number of grups.

N

Sample Size.

T.obs

Observed test statistic.

pvalue

P-value.

T.perm

Vector. Test statistics from the permutations.

n_perm

Number of permutations.

parameters

Estimated parameters.

sample_sizes

Groups lengths.

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

References

Chung, E. and Romano, J. P. (2013). Exact and asymptotically robust permutation tests. The Annals of Statistics, 41(2):484–507. Chung, E. and Romano, J. P. (2016). Asymptotically valid and exact permutation tests based on two-sample u-statistics. Journal of Statistical Planning and Inference, 168:97–105. Devroye, L. P. and Wagner, T. J. (1980). The strong uniform consistency of kernel density estimates. In Multivariate Analysis V: Proceedings of the fifth International Symposium on Multivariate Analysis, volume 5, pages 59–77. Efron, B. (1992). Bootstrap methods: another look at the jackknife. In Breakthroughs in statistics, pages 569–593. Springer. Hall, P., DiCiccio, T. J., and Romano, J. P. (1989). On smoothing and the bootstrap. The Annals of Statistics, pages 692–704. Hollander, M. (1967). Asymptotic efficiency of two nonparametric competitors of wilcoxon’s two sample test. Journal of the American Statistical Association, 62(319):939–949. Lehmann, E. L. (1951). Consistency and unbiasedness of certain nonparametric tests. The Annals of Mathematical Statistics, pages 165–179.

Examples

## Not run: 
male<-rnorm(50,1,1)
female<-rnorm(50,1,2)
dta<-data.frame(group=c(rep(1,50),rep(2,50)),outcome=c(male,female))
rpt.var<-RPT(dta$outcome~dta$group,test="variances")
summary(rpt.var)


## End(Not run)