frag.ma {fragility}

Assessing Fragility of a Meta-Analysis With a Binary Outcome

Description

Produces results, such as fragility index, fragility quotient, and event status modifications for altering statistical significance, of a meta-analysis with a binary outcome.

Usage

frag.ma(e0, n0, e1, n1, data, measure = "OR", alpha = 0.05,
        mod.dir = "both", OR = 1, RR = 1, RD = 0,
        method = "DL", test = "z", ...)

Arguments

e0

a numeric vector or the corresponding column name in the argument data, indicating event counts in group 0 among the studies in the meta-analysis.

n0

a numeric vector or the corresponding column name in the argument data, indicating sample sizes in group 0 among the studies in the meta-analysis.

e1

a numeric vector or the corresponding column name in the argument data, indicating event counts in group 1 among the studies in the meta-analysis.

n1

a numeric vector or the corresponding column name in the argument data, indicating sample sizes in group 1 among the studies in the meta-analysis.

data

an optional data frame containing the dataset of the collected studies in the meta-analysis with a binary outcome. If data is specified, the previous arguments, e0, n0, e1, and n1, should be specified as their corresponding column names in data.

measure

a character string indicating the measure of treatment effect (i.e., effect size) for the binary outcome. It should be one of "OR" (odds ratio, the default), "RR" (relative risk), and "RD" (risk difference).

alpha

a numeric value between 0 and 1, indicating the statistical significance level (the default is 0.05). It determines the confidence level (1-\alpha)\times100\% of the confidence interval when deriving the fragility index/quotient.

mod.dir

a character string indicating the direction of the confidence interval change due to event status modifications when the original confidence interval covers the null value (i.e., non-significance altered to significance). It is not used when significance is altered to non-significance. It should be one of "left" (moving to the left side of the null value), "right" (moving to the right side of the null value), "one" (based on the direction of the point estimate of the original overall effect size), and "both" (both directions, the default).

OR

a numeric positive value indicating the value of odds ratio under the null hypothesis (the default is 1). It is used only if the argument measure is "OR".

RR

a numeric positive value indicating the value of relative risk under the null hypothesis (the default is 1). It is used only if the argument measure is "RR".

RD

a numeric value between -1 indicating the value of risk difference under the null hypothesis (the default is 0). It is used only if the argument measure is "RD".

method

a character string specifying the method for performing meta-analysis. It is passed to rma.uni. Options include the fixed-effect model by specifying "FE" and the random-effects model by specifying "DL" (DerSimonian–Laird estimator), "HE" (Hedges estimator), "HS" (Hunter–Schmidt estimator), "SJ" (Sidik–Jonkman estimator), "ML" (maximum likelihood estimator), "REML" (restricted maximum likelihood estimator), "EB" (empirical Bayes estimator), "PM" (Paule–Mandel estimator), or "GENQ" (generalized W-statistic estimator) for different methods to estimate the heterogeneity variance. See details in the manual of the package metafor.

test

a character string specifying how confidence intervals are derived. It is passed to rma.uni. Options include "z", "t", "knha" (Hartung–Knapp–Sidik–Jonkman method), and "adhoc"; the first indicates Wald-type confidence intervals based on the standard normal distribution (the default), while the latter three yield confidence intervals based on the t-distribution. See details in the manual of the package metafor.

...

other arguments that can be passed to rma.uni. See details in the manual of the package metafor.

Value

An object of class "frag.ma". The object is a list containing the following components:

data

original data in the form of a data frame with six columns: event counts, non-event counts, and sample sizes in group 0 and those in group 1 among all studies in the meta-analysis.

measure

measure of treatment effect (i.e., effect size).

alpha

pre-specified statistical significance level.

null

value of odds ratio, relative risk, or risk difference (specified by measure) under the null hypothesis. Note that odds ratio and relative risk are presented on a natural logarithmic scale in all output values.

est.ori

the point estimate of the overall effect size based on the original meta-analysis.

ci.ori

the confidence interval of the overall effect size based on the original meta-analysis.

pval.ori

the p-value of the overall effect size based on the original meta-analysis.

mod.dir

the direction of the confidence interval change due to event status modifications when the original confidence interval covers the null value (i.e., non-significance altered to significance).

FI

fragility index.

FQ

fragility quotient.

dir

direction of the change of statistical significance.

sid.iter

study index(es) with event status modifications during the iterative algorithm.

g0.mod.iter

event status modification(s) in group 0 during the iterative algorithm.

g1.mod.iter

event status modification(s) in group 1 during the iterative algorithm.

est.iter

point estimate(s) of the overall effect size during the iterative algorithm.

ci.iter

confidence interval(s) of the overall effect size during the iterative algorithm.

data.mod

data with event status modification(s) that alter the significance or non-significance.

Note

If the significance or non-significance cannot be altered based on the given range of event status modifications, the fragility index and fragility quotient are defined to be NA (not available).

References

Atal I, Porcher R, Boutron I, Ravaud P (2019). "The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses." Journal of Clinical Epidemiology, 111, 32–40. <doi: 10.1016/j.jclinepi.2019.03.012>

Lin L, Chu H (2022). "Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package." PLOS ONE, 17(6), e0268754. <doi: 10.1371/journal.pone.0268754>

See Also

frag.ma.alpha for assessing fragility of a meta-analysis at multiple significance levels; frag.mas for assessing fragility of multiple meta-analyses; frag.study for assessing fragility of an individual study; and frag.nma for assessing fragility of a network meta-analysis.

Examples

## Load datasets of meta-analyses on nutrition support
data(dat.ns)

## Assess fragility of the 11th meta-analysis
out1 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,])
out1

## Visualize the process of event status modifications
##  for altering the non-significance to significance
plot(out1)

## Assess fragility based on relative risk
out2 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  measure = "RR")
out2

## Assess fragility based on risk difference
out3 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  measure = "RD")
out3

## Use the REML method to estimate the heterogeneity variance
out4 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  method = "REML")
out4

## Use the Hartung-Knapp-Sidik-Jonkman to derive
##  the confidence interval in meta-analysis
out5 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  method = "REML", test = "knha")
out5

## Change the significance level to 0.005
out6 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 11,],
  alpha = 0.005)
out6
plot(out6)

## Reduce the distance between the two lines by breaking the y-axis
plot(out6, ybreaks = c(5, 9))

## Suppress the markers of study IDs around the lines
plot(out6, ybreaks = c(5, 9), study.marker = FALSE)

## Change the size of markers of study IDs
plot(out6, ybreaks = c(5, 9), cex.marker = 0.5)

## Change the distance between the markers of study IDs and the lines
plot(out6, ybreaks = c(5, 9), offset.marker = 0)

## Change the colors and width of the lines
plot(out6, ybreaks = c(5, 9), col.line = c("gray", "orange"), lwd = 2)

## Change the location of the legend
plot(out6, ybreaks = c(5, 9), x.legend = "topleft")


## Assess fragility of the first meta-analysis;
##  it takes longer time
out7 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 1,])
out7
plot(out7)
plot(out7, ybreaks = c(840, 880))

## Achieve significance by moving the CI based on the direction of the 
##  original overall effect size estimate
out8 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 1,],
  mod.dir = "one")
out8

## Achieve significance by moving the CI to the left side
out9 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 1,],
  mod.dir = "left")
out9

## Achieve significance by moving the CI to the right side;
##  it takes longer time than that moving the CI to the left side
##  as the original overall effect size estimate was less than the null
out10 <- frag.ma(e0, n0, e1, n1, data = dat.ns[dat.ns$ma.id == 1,],
  mod.dir = "right")
out10
dev.new(width = 12, height = 6)
plot(out10, cex.marker = 0.5)