| dfba_power_curve {DFBA} | R Documentation |
Power Curves
Description
This function is a design tool for comparing Bayesian distribution-free power with frequentist t power for a range of delta values, which are the separation values between two continuous variates that can be randomly sampled from one of nine different probability models. The function provides a table of power values for a fixed sample size value of n.
Usage
dfba_power_curve(
n = 20,
a0 = 1,
b0 = 1,
delta_step = 0.05,
model,
design,
effect_crit = 0.95,
shape1 = 1,
shape2 = 1,
block_max = 0,
samples = 1000,
hide_progress = FALSE
)
Arguments
n |
The sample size for both variates (default is 20) |
a0 |
The first shape parameter for the prior beta distribution (default is 1). Must be positive and finite. |
b0 |
The second shape parameter for the prior beta distribution (default is 1). Must be positive and finite. |
delta_step |
The increment between successive delta values, which range from 0 to |
model |
Theoretical probability model for the data. One of |
design |
Indicates the data structure. One of |
effect_crit |
Stipulated value for a significant differences for a t-test (1 - p), and the critical probability for the Bayesian alternative hypothesis for a Bayesian distribution-free analysis |
shape1 |
The shape parameter for the condition 1 variate for the distribution indicated by the |
shape2 |
The shape parameter for the condition 2 variate for the distribution indicated by the |
block_max |
The maximum size for a block effect (default is 0) |
samples |
Desired number of Monte Carlo data sets drawn to estimate the power (default is 1000) |
hide_progress |
(Optional) If |
Details
Researchers need to make experimental-design decisions such as the choice
about the sample size per condition and the decision of whether to use a
within-block design or an independent-group design. These planning issues
arise regardless if one uses either a frequentist or Bayesian approach to
statistical inference. In the DFBA package there are a number of functions to
help users with these decisions. The dfba_power_curve() function
produces the Bayesian power estimate from a distribution-free analysis along
with the corresponding frequentist power from a parametric t-test, for
21 delta values, which range from 0 to 20*delta_step where delta is the
separation between two random variates. The sample size for the power
estimates is the same value n in each condition. The power estimates
are based on a number of Monte Carlo sampled data sets generated by the
dfba_sim_data() function.
For each data set, statistical tests are performed. If design = "paired",
the frequentist t-test is a one-tailed test on the within-block
difference scores to assess the null hypothesis that the population mean for
E is greater than the population mean for C; if
design = "independent", the frequentist t-test is the one-tailed
test to assess if there is a significant difference between the two
independent conditions (i.e. if the mean for condition 2 is signficantly
greater than the condition 1 mean). If design = "paired", the Bayesian
analysis assesses if the posterior probability for phi_w > .5 on the
Bayesian Wilcoxon test is greater than effect_crit; if
design = "independent", the Bayesian analysis assesses if the posterior
probability for omega_E > .5 on a Bayesian Mann-Whitney test is greater
than effect_crit. The frequentist power is estimated by the proportion
of the data sets where a parametric t-test detects a significant effect
because the upper-tail t value has a p-value less than
1-effect_crit. The Bayesian power is the proportion of the data sets
where a posterior probability for the alternative hypothesis is greater than
effect_crit. The default value for the effect_crit argument is
.95. The frequentist p-value and the Bayesian posterior probability for
the alternative hypothesis are calculated using the dfba_sim_data()
function.
The arguments for the dfba_power_curve() function are passed into the
dfba_sim_data() function. Besides the sample size n, there
are eight other arguments that are required by the dfba_power_curve()
function, which are passed into the dfba_sim_data() function:
-
a0 -
b0 -
model -
design -
delta -
shape1 -
shape2 -
block_max.
The a0 and b0 values are the respective first and second beta
shape parameters for the prior distribution needed for the Bayesian
distribution-free tests, which are ultimately done by calling either the
dfba_wilcoxon() function or by the dfba_mann_whitney().
The model argument is one of the following strings:
-
"normal" -
"weibull" -
"cauchy" -
"lognormal" -
"chisquare" -
"logistic" -
"exponential" -
"gumbel" -
"pareto"
The design argument is either "independent" or "paired",
and stipulates whether the two sets of scores are either independent or from
a common block such as for the case of two scores for the same person (i.e.,
one in each condition).
The shape1 and shape2 arguments are values for the shape parameter
for the respective first and second condition, and their meaning
depends on the probability model. For model="normal", these
parameters are the standard deviations of the two distributions. For
model = "weibull", the parameters are the Weibull shape parameters.
For model = "cauchy", the parameters are the scale factors for the
Cauchy distributions. For model = "lognormal", the shape
parameters are the standard deviations for log(X). For model = "chisquare",
the parameters are the degrees of freedom (df) for the two
distributions. For model = "logistic", the parameters are the scale
factors for the distributions. For model = "exponential", the parameters
are the rate parameters for the distributions.
For the Gumbel distribution, the E variate is equal to
delta - shape2*log(log(1/U)) where U is a random value sampled
from the uniform distribution on the interval [.00001, .99999], and
the C variate is equal to -shape1*log(log(1/U)) where U
is another score sampled from the uniform distribution. The shape1 and
shape2 arguments for model = "gumbel" are the scale parameters
for the distributions. The Pareto model is a distribution designed to account
for income distributions as studied by economists (Pareto, 1897). For the
Pareto distribution, the cumulative function is equal to 1-(x_m/x)^alpha
where x is greater than x_m (Arnold, 1983). In the E
condition, x_m = 1 + delta and in the C condition x_m = 1.
The alpha parameter is 1.16 times the shape parameters shape1 and
shape2. Since the default value for each shape parameter is 1, the
resulting alpha value of 1.16 is the default value. When alpha = 1.16, the
Pareto distribution approximates an income distribution that represents the
80-20 law where 20% of the population receives 80% of the income
(Hardy, 2010).
The block_max argument provides for incorporating block effects in the
random sampling. The block effect for each score is a separate effect for the
block. The block effect B for a score is a random number drawn from a uniform
distribution on the interval [0, block_max]. When design = "paired",
the same random block effect is added to the score in the first condition,
which is the random C value, and it is also added to the corresponding
paired value for the E variate. Thus, the pairing research design
eliminates the effect of block variation for the assessment of condition
differences. When design = "independent", there are different block-effect
contributions to the E and C variates, which reduces the
discrimination of condition differences because it increases the variability
of the difference in the two variates. The user can study the effect of the
relative discriminability of detecting an effect of delta by adjusting the
value of the block_max argument. The default for block_max is 0,
but it can be altered to any non-negative real number.
Value
A list containing the following components:
n |
The fixed sample size for each variate |
nsims |
The number of Monte Carlo data sets; equal to the value of the |
model |
Probability model for the data |
design |
The design for the data; one of |
a0 |
The first shape parameter for the beta prior distribution for the Bayesian analysis |
b0 |
The second shape parameter for the beta prior distribution for the Bayesian analysis |
effect_crit |
The criterion probability for considering a posterior probability for the alternative hypothesis to be a detection; it is also |
block_max |
The maximum size of a block effect; equal to the |
delta_vec |
Vector of the 21 delta offset values |
Bayes_power |
The vector of 21 Bayesian power values |
t_power |
The vector of 21 frequentist power from t-tests |
outputdf |
A dataframe for the Bayesian power and the corresponding frequentist t power as a function of delta |
References
Arnold, B. C. (1983). Pareto Distribution. Fairland, MD: International Cooperative Publishing House.
Chechile, R. A. (2017). A Bayesian analysis for the Wilcoxon signed-rank statistic. Communications in Statistics - Theory and Methods, https://doi.org/10.1080/03610926.2017.1388402
Chechile, R. A. (2020). A Bayesian analysis for the Mann-Whitney statistic. Communications in Statistics - Theory and Methods, https://doi.org/10.1080/03610926.2018.1549247
Fishman, G. S. (1996) Monte Carlo: Concepts, Algorithms and Applications. New York: Springer.
Hardy, M. (2010). Pareto's Law. Mathematical Intelligencer, 32, 38-43.
Johnson, N. L., Kotz S., and Balakrishnan, N. (1995). Continuous Univariate Distributions, Vol. 1, New York: Wiley.
Pareto, V. (1897). Cours d'Economie Politique. Vol. 2, Lausanne: F. Rouge.
See Also
Distributions for details on the
parameters of the normal, Weibull, Cauchy, lognormal, chi-squared, logistic,
and exponential distributions.
dfba_sim_data for further details about the data for two
conditions that differ in terms of their theoretical mean by an amount delta.
Examples
# Note: these examples have long runtimes due to Monte Carlo sampling;
# please feel free to run them in the console.
dfba_power_curve(n = 85,
model = "normal",
design = "independent",
samples = 250,
hide_progress = TRUE)
dfba_power_curve(n = 85,
model = "normal",
design = "paired",
samples = 250,
hide_progress = TRUE)
# Using the Jeffreys prior rather than default flat prior
dfba_power_curve(n = 30,
model = "lognormal",
design = "independent",
a0 = .5,
b0 = .5,
delta_step = .06,
block_max = 3,
samples = 250,
hide_progress = TRUE)