CPValidity {conformalClassification}

Computes the deviation from exact validity as the Euclidean norm of the difference of the observed error and the expected error

Description

Computes the deviation from exact validity as the Euclidean norm of the difference of the observed error and the expected error

Usage

CPValidity(matPValues = NULL, testLabels = NULL)

Arguments

matPValues

Matrix of p-values

testLabels

True labels for the test-set

Value

The deviation from exact validity

See Also

CPCalibrationPlot, CPEfficiency, CPErrorRate, CPObsFuzziness.

Examples

## load the library
library(mlbench)
#library(caret)
library(conformalClassification)

## load the DNA dataset
data(DNA)
originalData <- DNA

## make sure first column is always the label and class labels are always 1, 2, ...
nrAttr = ncol(originalData) #no of attributes
tempColumn = originalData[, 1]
originalData[, 1] = originalData[, nrAttr]
originalData[, nrAttr] = tempColumn
originalData[, 1] = as.factor(originalData[, 1])
originalData[, 1] = as.numeric(originalData[, 1])

## partition the data into training and test set
#result = createDataPartition(originalData[, 1], p = 0.8, list = FALSE)
size = nrow(originalData)
result = sample(1:size,  0.8*size)

trainingSet = originalData[result, ]
testSet = originalData[-result, ]

##ICP classification
pValues = ICPClassification(trainingSet, testSet)
testLabels = testSet[,1]
CPValidity(pValues, testLabels)