StSignificanceTesting {RJafroc}

Performs DBM or OR significance testing for factorial or split-plot A,C datasets

Description

Performs Dorfman-Berbaum-Metz (DBM) or Obuchowski-Rockette (OR) significance testing, for specified dataset; significance testing refers to analysis designed to assign a P-value, and other statistics, for rejecting the null hypothesis (NH) that the reader-averaged figure of merit (FOM) differences between treatments is zero. The results of the analysis are best visualized in the text or Excel-formatted files produced by UtilOutputReport.

Usage

StSignificanceTesting(
  dataset,
  FOM,
  FPFValue = 0.2,
  alpha = 0.05,
  method = "DBM",
  covEstMethod = "jackknife",
  nBoots = 200,
  analysisOption = "ALL",
  tempOrgCode = FALSE
)

Arguments

dataset

The dataset to be analyzed, see RJafroc-package. Must have two or more treatments and two or more readers. The dataset design can be "FCTRL", "SPLIT-PLOT-A" or "SPLIT-PLOT-C".

FOM

The figure of merit, see UtilFigureOfMerit

FPFValue

Only needed for LROC data and FOM = "PCL" or "ALROC"; where to evaluate a partial curve based figure of merit. The default is 0.2.

alpha

The significance level of the test of the null hypothesis that all treatment effects are zero; the default is 0.05

method

The significance testing method to be used: "DBM" (the default), representing the Dorfman-Berbaum-Metz method or "OR", representing the Obuchowski-Rockette method. and the Obuchowski-Rockette significance testing methods, respectively.

covEstMethod

The covariance matrix estimation method in ORH analysis (for method = "DBM" the jackknife is always used).

• "Jackknife", the default,

• "Bootstrap", in which case nBoots (above) is relevant,

• "DeLong"; requires FOM = "Wilcoxon" or "ROI" or "HrAuc", otherwise an error results.

nBoots

The number of bootstraps (defaults to 200), relevant only if covEstMethod = "bootstrap" and method = "OR"

analysisOption

Determines which factors are regarded as random vs. fixed:

• "RRRC" = random-reader random case,

• "FRRC" = fixed-reader random case,

• "RRFC" = random-reader fixed case,

• "ALL" = outputs results of "RRRC", "FRRC" and "RRFC" analyses - this is the default.

tempOrgCode

default FALSE; if TRUE, then code from version 0.0.1 of RJafroc is used (see RJafroc_0.0.1.tar). This is intended to check against errors that crept in subsequent to the original version as I attempted to improve the organization of the code and the output. As implicit in the name of this temporary flag, it will eventually be removed.

Value

For method = "DBM" the returned list contains 4 dataframes:

FOMs

Contains foms, trtMeans and trtMeanDiffs: see return of UtilFigureOfMerit

ANOVA

Contains TRCAnova, VarCom, IndividualTrt and IndividualRdr ANOVA tables of pseudovalues

RRRC

Contains results of "RRRC" analyses: FTests, ciDiffTrt, ciAvgRdrEachTrt

FRRC

Contains results of "FRRC" analyses: FTests, ciDiffTrt, ciAvgRdrEachTrt, ciDiffTrtEachRdr

RRFC

Contains results of "RRFC" analyses: FTests, ciDiffTrt, ciAvgRdrEachTrt

For method = "OR" the return list contains 4 dataframes:

FOMs

Contains foms, trtMeans and trtMeanDiffs: UtilFigureOfMerit

ANOVA

Contains TRAnova, VarCom, IndividualTrt and IndividualRdr ANOVA tables of FOM values

RRRC

Contains results of "RRRC" analyses - same organization as DBM, see above

FRRC

Contains results of "FRRC" analyses - ditto

RRFC

Contains results of "RRFC" analyses- ditto

References

Dorfman DD, Berbaum KS, Metz CE (1992) ROC characteristic rating analysis: Generalization to the Population of Readers and Patients with the Jackknife method, Invest. Radiol. 27, 723-731.

Obuchowski NA, Rockette HE (1995) Hypothesis Testing of the Diagnostic Accuracy for Multiple Diagnostic Tests: An ANOVA Approach with Dependent Observations, Communications in Statistics: Simulation and Computation 24, 285-308.

Hillis SL (2014) A marginal-mean ANOVA approach for analyzing multireader multicase radiological imaging data, Statistics in medicine 33, 330-360.

Chakraborty DP (2017) Observer Performance Methods for Diagnostic Imaging - Foundations, Modeling, and Applications with R-Based Examples, CRC Press, Boca Raton, FL. https://www.routledge.com/Observer-Performance-Methods-for-Diagnostic-Imaging-Foundations-Modeling/Chakraborty/p/book/9781482214840

Examples

StSignificanceTesting(dataset02,FOM = "Wilcoxon", method = "DBM") 
StSignificanceTesting(dataset02,FOM = "Wilcoxon", method = "OR")
## following is split-plot-c analysis using a simulated split-plot-c dataset
StSignificanceTesting(datasetFROCSpC, FOM = "wAFROC", method = "OR")


StSignificanceTesting(dataset05, FOM = "wAFROC")
StSignificanceTesting(dataset05, FOM = "HrAuc", method = "DBM") 
StSignificanceTesting(dataset05, FOM = "SongA1", method = "DBM") 
StSignificanceTesting(dataset05, FOM = "SongA2", method = "DBM") 
StSignificanceTesting(dataset05, FOM = "wAFROC1", method = "DBM")
StSignificanceTesting(dataset05, FOM = "AFROC1", method = "DBM")
StSignificanceTesting(dataset05, FOM = "AFROC", method = "DBM")