frag.study.alpha {fragility}

Assessing Fragility of an Individual Study at Different Significance Levels

Description

Produces fragility index or fragility quotient for altering statistical significance of an individual study with a binary outcome at different significance levels.

Usage

frag.study.alpha(e0, n0, e1, n1, data, methods,
                 modify0 = "both", modify1 = "both",
                 alpha.from = 0.005, alpha.to = 0.05, alpha.breaks = 100,
                 alternative = "two.sided", OR = 1, RR = 1, RD = 0)

Arguments

e0

a numeric value or the corresponding column name in the argument data, indicating event count in group 0.

n0

a numeric value or the corresponding column name in the argument data, indicating sample size in group 0.

e1

a numeric value or the corresponding column name in the argument data, indicating event count in group 1.

n1

a numeric value or the corresponding column name in the argument data, indicating sample size in group 1.

data

an optional data frame containing the dataset of the individual study with a binary outcome. If data is specified, the previous arguments, e0, n0, e1, and n1, should be specified as their corresponding column names in data.

methods

a character string or a vector of character strings indicating the method(s) used to calculate the p-value(s) of the association between treatment and outcome. It must consist of "Fisher" (Fisher's exact test), "chisq" (chi-squared test), "OR" (odds ratio), "RR" (relative risk), and/or "RD" (risk difference). The default is the vector consisting of all aforementioned five methods.

modify0

a character string indicating the event status modifications in group 0. It should be one of "increase" (increasing event numbers), "decrease" (decreasing event numbers), "both" (the default, modifying event numbers in both directions), and "none" (no modification).

modify1

a character string indicating the event status modifications in group 1. It has the same usage with modify0, with the default being "both".

alpha.from

a numeric value between 0 and 1, indicating the smallest value of the statistical significance levels to be considered (the default is 0.005). It should be smaller than the argument alpha.to.

alpha.to

a numeric value between 0 and 1, indicating the largest value of the statistical significance levels to be considered (the default is 0.05). It should be larger than the argument alpha.from.

alpha.breaks

a positive integer indicating the number of statistical significance levels to be considered (the default is 100). The candidate significance levels are thus equally-spaced between alpha.from and alpha.to.

alternative

a character string of either "two.sided" (the default) or "one.sided", indicating the alternative hypothesis. It is only used for "OR", "RR", and "RD" in the argument methods.

OR

a numeric positive value indicating the value of odds ratio under the null hypothesis (the default is 1). It is used only if the argument methods includes "OR".

RR

a numeric positive value indicating the value of relative risk under the null hypothesis (the default is 1). It is used only if the argument methods includes "RR".

RD

a numeric value between -1 indicating the value of risk difference under the null hypothesis (the default is 0). It is used only if the argument methods includes "RD".

Value

An object of classes "frag.alpha" and "frag.study.alpha". The object is a list containing the following components:

data

original data in the form of 2\times2 table.

methods

methods used to calculate the p-value(s) of the association between treatment and outcome.

alphas

different statistical significance levels.

alternative

alternative hypothesis.

null

values of odds ratio, relative risk, and risk difference under the null hypothesis; they are only used if "OR", "RR", and "RD" are included in the argument methods, respectively.

modify0

type of event status modifications in group 0.

modify1

type of event status modifications in group 1.

f0.range

range of event status modifications in group 0.

f1.range

range of event status modifications in group 0.

pval

p-value(s) produced by the method(s) included in the argument methods.

FI

fragility indexes based on the method(s) at each statistical significance level in alphas.

FI.avg

average fragility index based on each method.

FQ

fragility quotients based on the method(s) at each statistical significance level in alphas.

FQ.avg

average fragility quotient based on each method.

Of note, if both arguments modify0 and modify0 are "none" (i.e., no modification), the items following pval in the above list are not produced, because fragility cannot be assessed without event status modifications.

References

Benjamin DJ, Berger JO, Johannesson M, Nosek BA, Wagenmakers EJ, Berk R, Bollen KA, Brembs B, Brown L, Camerer C, Cesarini D, Chambers CD, Clyde M, Cook TD, De Boeck P, Dienes Z, Dreber A, Easwaran K, Efferson C, Fehr E, Fidler F, Field AP, Forster M, George EI, Gonzalez R, Goodman S, Green E, Green DP, Greenwald AG, Hadfield JD, Hedges LV, Held L, Ho TH, Hoijtink H, Hruschka DJ, Imai K, Imbens G, Ioannidis JPA, Jeon M, Jones JH, Kirchler M, Laibson D, List J, Little R, Lupia A, Machery E, Maxwell SE, McCarthy M, Moore DA, Morgan SL, Munafo M, Nakagawa S, Nyhan B, Parker TH, Pericchi L, Perugini M, Rouder J, Rousseau J, Savalei V, Schonbrodt FD, Sellke T, Sinclair B, Tingley D, Van Zandt T, Vazire S, Watts DJ, Winship C, Wolpert RL, Xie Y, Young C, Zinman J, Johnson VE (2018). "Redefine statistical significance." Nature Human Behaviour, 2, 6–10. <doi: 10.1038/s41562-017-0189-z>

Ioannidis JPA (2018). "The proposal to lower P value thresholds to .005." JAMA, 319(14), 1429–30. <doi: 10.1001/jama.2018.1536>

Lin L, Chu H (2022). "Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package." PLOS ONE, 17(6), e0268754. <doi: 10.1371/journal.pone.0268754>

See Also

frag.study for assessing fragility of an individual study at a specific significance level.

Examples

## Load datasets of trials on antidepressant drugs
data(dat.ad)

## Assess fragility of trial 13 at multiple significance levels
out1 <- frag.study.alpha(e0, n0, e1, n1, data = dat.ad[13,])
out1

## Generate plots to show the change of fragility measure,
##  where the default plot is for the first method
##  (i.e., Fisher's exact test) if the argument method is not specified
plot(out1)

## The plot for odds ratio
plot(out1, method = "OR")

## Change the choices of significance levels
out2 <- frag.study.alpha(e0, n0, e1, n1, data = dat.ad[13,],
  methods = "OR", alpha.from = 0.001, alpha.to = 0.1, alpha.breaks = 500)
out2
plot(out2)

## Generate the plot with the x-axis on the log scale
plot(out2, log = "x")

## Generate the plot for fragility quotient (in percentage)
plot(out2, fragility = "FQ", log = "x")

## Generate the plot for fragility quotient (in decimal form)
plot(out2, fragility = "FQ", percentage = FALSE, log = "x")

## Change the size of points
plot(out2, cex.pts = 1, log = "x")

## Change the colors
plot(out2, col.line = "blue", col.pval = "orange",
  col.sig = c("gray", "black"), log = "x")

## Change the line type
plot(out2, lty.pval = 3, log = "x")

## Change the line width
plot(out2, lwd = 2, lwd.pval = 2, log = "x")

## Change the point symbol
plot(out2, pch = 1, log = "x")