topPerformer {performanceEstimation} | R Documentation |
Obtain the workflow that best performed in terms of a metric on a task
Description
This function can be used to obtain the workflow (an object of class
Workflow
) that performed better in terms of a
given metric on a certain task.
Usage
topPerformer(compRes,metric,task,max=FALSE,stat="avg")
Arguments
compRes |
A |
metric |
A string with the name of a metric estimated in the comparison |
task |
A string with the name of a predictive task used in the comparison |
max |
A boolean (defaulting to |
stat |
The statistic to be used to obtain the ranks. The options are the
statistics produced by the function |
Details
This is an utility function that can be used to obtain the workflow
(an object of class Workflow
) that achieved the
best performance on a given predictive task in terms of a certain
evaluation metric. The notion of best performance depends on
the type of evaluation metric, thus the need for the max
argument. Some evaluation statistics are to be maximized
(e.g. accuracy), while others are to be minimized (e.g. mean squared
error). For the former you should use max=TRUE
, while the
latter require max=FALSE
(the default).
Value
The function returns an object of class Workflow
.
Author(s)
Luis Torgo ltorgo@dcc.fc.up.pt
References
Torgo, L. (2014) An Infra-Structure for Performance Estimation and Experimental Comparison of Predictive Models in R. arXiv:1412.0436 [cs.MS] http://arxiv.org/abs/1412.0436
See Also
performanceEstimation
,
topPerformers
,
rankWorkflows
,
metricsSummary
Examples
## Not run:
## Estimating several evaluation metrics on different variants of a
## regression tree and of a SVM, on two data sets, using one repetition
## of 10-fold CV
data(swiss)
data(mtcars)
library(e1071)
## run the experimental comparison
results <- performanceEstimation(
c(PredTask(Infant.Mortality ~ ., swiss),
PredTask(mpg ~ ., mtcars)),
c(workflowVariants(learner='svm',
learner.pars=list(cost=c(1,5),gamma=c(0.1,0.01))
)
),
EstimationTask(metrics=c("mse","mae"),method=CV(nReps=2,nFolds=5))
)
## get the top performer workflow for a metric and task
topPerformer(results,"mse","swiss.Infant.Mortality")
## End(Not run)