predefinedObjectiveFunctions {TunePareto} | R Documentation |
Predefined objective functions for parameter tuning
Description
Predefined objective functions that calculate several performance criteria of reclassification or cross-validation experiments.
Usage
reclassAccuracy(saveModel = FALSE)
reclassError(saveModel = FALSE)
reclassWeightedError(saveModel = FALSE)
reclassSensitivity(caseClass, saveModel = FALSE)
reclassRecall(caseClass, saveModel = FALSE)
reclassTruePositive(caseClass, saveModel = FALSE)
reclassSpecificity(caseClass, saveModel = FALSE)
reclassTrueNegative(caseClass, saveModel = FALSE)
reclassFallout(caseClass, saveModel = FALSE)
reclassFalsePositive(caseClass, saveModel = FALSE)
reclassMiss(caseClass, saveModel = FALSE)
reclassFalseNegative(caseClass, saveModel = FALSE)
reclassPrecision(caseClass, saveModel = FALSE)
reclassPPV(caseClass, saveModel = FALSE)
reclassNPV(caseClass, saveModel = FALSE)
reclassConfusion(trueClass, predictedClass, saveModel = FALSE)
cvAccuracy(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified = FALSE,
foldList = NULL,
saveModel = FALSE)
cvError(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL,
saveModel = FALSE)
cvErrorVariance(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL,
saveModel = FALSE)
cvWeightedError(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL,
saveModel = FALSE)
cvSensitivity(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvRecall(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvTruePositive(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvSpecificity(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvTrueNegative(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvFallout(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvFalsePositive(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvMiss(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvFalseNegative(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvPrecision(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvPPV(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvNPV(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, caseClass,
saveModel = FALSE)
cvConfusion(nfold = 10, ntimes = 10,
leaveOneOut = FALSE, stratified=FALSE,
foldList=NULL, trueClass, predictedClass,
saveModel = FALSE)
Arguments
nfold |
The number of groups of the cross-validation. Ignored if |
ntimes |
The number of repeated runs of the cross-validation. Ignored if |
leaveOneOut |
If this is true, a leave-one-out cross-validation is performed, i.e. each sample is left out once in the training phase and used as a test sample |
stratified |
If set to true, a stratified cross-validation is carried out. That is, the percentage of samples from different classes in the cross-validation folds corresponds to the class sizes in the complete data set. If set to false, the folds may be unbalanced. |
foldList |
If this parameter is set, the other cross-validation parameters ( |
caseClass |
The class containing the positive samples for the calculation of specificity and sensitivity. All samples with different class labels are regarded as controls (negative samples). |
trueClass |
When calculating the confusion of two classes, the class to which a sample truly belongs. |
predictedClass |
When calculating the confusion of two classes, the class to which a sample is erreneously assigned. |
saveModel |
If set to true, the trained model(s) are stored to the |
Details
The functions do not calculate the objectives directly, but return a structure of class TuneParetoObjectives
that provides all information on the objective function for later use in tunePareto
.
The behaviour of the functions in tunePareto
is as follows:
The reclassification functions train the classifiers with the full data set. Afterwards, the classifiers are applied to the same data set. reclassAccuracy
measures the fraction of correctly classified samples, while reclassError
calculates the fraction of misclassified samples. reclassWeightedError
calculates the sum of fractions of misclassified samples in each class weighted by the class size. reclassSensitivity
measures the sensitivity, and reclassSpecificity
measures the specificity of the reclassification experiment. reclassTruePositive
and reclassRecall
are aliases for reclassSensitivity
, and reclassTrueNegative
is an alias for reclassSpecificity
. reclassFallout
and its equivalent alias reclassFalsePositive
give the ratio of false positives to all negative samples, and reclassMiss
and its alias reclassFalseNegative
measure the ratio of false negatives to all positive samples. reclassPrecision
calculates the precision of the reclassification experiment, i.e. the ratio of true positives to all samples classified as positive. This is equivalent to the positive predictive value (reclassPPV
). reclassNPV
measures the negative predictive value, i.e. the ratio of true negatives to all samples classified as negative. reclassConfusion
calculates the fraction of samples in trueClass
that have been confused with predictedClass
.
reclassError
, reclassWeightedError
, reclassFallout
, reclassFalsePositive
, reclassMiss
, reclassFalsePositive
and reclassConfusion
are minimization objectives, whereas reclassAccuracy
, reclassSensitivity
, reclassTruePositive
, reclassRecall
, reclassSpecificity
, reclassTrueNegative
reclassPrecision
, reclassPPV
and reclassNPV
are maximization objectives.
The cross-validation functions partition the samples in the data set into a number of groups (depending on nfold
and leaveOneOut
). Each of these groups is left out once in the training phase and used for prediction. The whole procedure is usually repeated several times (as specified in ntimes
), and the results are averaged. Similar to the reclassification functions, cvAccuracy
calculates the fraction of correctly classified samples over the runs, cvError
calculates the average fraction of misclassified samples over the runs, and cvWeightedError
calculates the mean sum of fractions of misclassified samples in each class weighted by the class size. cvErrorVariance
calculates the variance of the cross-validation error. cvSensitivity
, cvRecall
and cvTruePositive
calculate the average sensitivity, and cvSpecificity
and cvTrueNegative
calculate the average specificity. cvFallout
and cvFalsePositive
calculate the average false positive rate over the runs. cvMiss
and cvFalseNegative
calculate the average false negative rate over the runs. cvPrecision
and cvPPV
calculate the average precision/positive predictive rate. cvNPV
gives the average negative predictive rate over all runs. cvConfusion
calculates the average fraction of samples in trueClass
that have been confused with predictedClass
.
cvError
, cvWeightedError
, cvErrorVariance
, cvFallout
, cvFalsePositive
, cvMiss
, cvFalseNegative
and cvConfusion
are minimization objectives, and cvAccuracy
, cvSensitivity
, cvRecall
, cvTruePositive
, cvSpecificity
, cvTrueNegative
, cvPrecision
, cvPPV
and cvNPV
are maximization objectives.
Value
An object of class TuneParetoObjective
representing the objective function. For more details, see createObjective
.
See Also
createObjective
, tunePareto
, generateCVRuns
Examples
# build a list of objective functions
objectiveFunctions <- list(cvError(10, 10),
reclassSpecificity(caseClass="setosa"),
reclassSensitivity(caseClass="setosa"))
# pass them to tunePareto
print(tunePareto(data = iris[, -ncol(iris)],
labels = iris[, ncol(iris)],
classifier = tunePareto.knn(),
k = c(3,5,7,9),
objectiveFunctions = objectiveFunctions))