robustness_index {rnmamod}

Robustness index

Description

Calculates the robustness index, a novel index that quantifies the overall divergence of the sensitivity analysis results from the primary analysis results. The robustness index considers objective decision rules to infer the presence or lack of robustness of the primary analysis results when conducting a sensitivity analysis (Spineli et al., 2021).

Usage

robustness_index(sens, threshold)

Arguments

sens

An object of S3 class run_sensitivity when sensitivity analysis refers to different scenarios about the average missingness parameter. See 'Value' in run_sensitivity. For a general sensitivity analysis, insert a list of at least two objects of S3 class run_model indicating different re-analyses: the first object (of class run_model) in the list should refer to the primary analysis.

threshold

A number indicating the threshold of robustness, that is, the minimally allowed deviation between the primary analysis and re-analysis results. See 'Details' below.

Details

Thresholds of robustness have been proposed only for the odds ratio and standardised mean difference (Spineli et al., 2021). The user may consider the values 0.28 and 0.17 in the argument threshold for the odds ratio and standardised mean difference effect measures (the default values), respectively, or consider other plausible values. When the argument threshold has not been defined, robustness_index considers the default values 0.28 and 0.17 as threshold for robustness for binary and continuous outcome, respectively, regardless of the effect measure (the default thresholds may not be proper choices for other effect measures; hence, use these threshold with great caution in this case). Spineli et al. (2021) offers a discussion on specifying the threshold of robustness.

In the case of binary outcome, robustness_index considers the results in the odds ratio scale to calculate the robustness index. This is because, the odds ratio is used as the 'best-case' effect measure in run_model. Then, relative risk, and risk difference are functions of the odds ratio and the selected baseline risk (See 'Details' in run_model).

In the case of missing participant outcome data, the primary analysis is considered to be the middle of the numbers in the argument mean_scenarios of run_sensitivity (see 'Arguments' and 'Details' in run_sensitivity).

In robust, the value "robust" appears when robust_index is less than threshold; otherwise, the value "frail" appears.

In the case of missing participant outcome data, robustness_index can be used only when missing participant outcome data have been extracted for at least one trial. Otherwise, the execution of the function will be stopped and an error message will be printed in the R console.

Value

robustness_index prints on the R console a message in green text on the threshold of robustness determined by the user. Then, the function returns the following list of elements:

robust_index

A numeric scalar or vector on the robustness index values. In the case of a pairwise meta-analysis, robust_index is scalar as only one summary effect size is obtained. In the case of network meta-analysis, robust_index is a vector with length equal to the number of possible pairwise comparisons; one robustness index per possible comparison.

robust

A character or character vector (of same length with robust_index) on whether the primary analysis results are robust or frail to the different re-analyses.

kld

A vector or matrix on the Kullback-Leibler divergence (KLD) measure in the summary effect size from a subsequent re-analysis to the primary analysis. In the case of a pairwise meta-analysis, kld is a vector with length equal to the number of total analyses (one KLD value is obtained per analysis). The number of total analyses equals the square of the number of scenarios indicated in the argument mean_scenarios of run_sensitivity, in the case of missing participant outcome data; otherwise, the length of the character vector in argument sens. In the case of network meta-analysis, robust_index is a matrix with number of rows equal to the number of total analyses and number of columns equal to the number of possible pairwise comparisons; one KLD value per analysis and possible comparison.

threshold

The threshold used to be inherited by the heatmap_robustness function.

scenarios

The scenarios considered to be inherited by the kld_barplot function.

Author(s)

Loukia M. Spineli

References

Kullback S, Leibler RA. On information and sufficiency. Ann Math Stat 1951;22(1):79–86. doi: 10.1214/aoms/1177729694

Spineli LM, Kalyvas C, Papadimitropoulou K. Quantifying the robustness of primary analysis results: A case study on missing outcome data in pairwise and network meta-analysis. Res Synth Methods 2021;12(4):475–90. doi: 10.1002/jrsm.1478

See Also

heatmap_robustness, kld_barplot, run_model, run_sensitivity

Examples

data("nma.baker2009")

# Read results from 'run_sensitivity' (using the default arguments)
res_sens <- readRDS(system.file('extdata/res_sens_baker.rds',
                    package = 'rnmamod'))

# Calculate the robustness index
robustness_index(sens = res_sens,
                 threshold = 0.28)