An Infra-Structure for Performance Estimation of Predictive Models

---

Documentation for package ‘performanceEstimation’ version 1.1.0

• DESCRIPTION file.
• User guides, package vignettes and other documentation.

Help Pages

bootEstimates	Performance estimation using (e0 or .632) bootstrap
Bootstrap	Class "Bootstrap"
Bootstrap-class	Class "Bootstrap"
CDdiagram.BD	CD diagrams for the post-hoc Boferroni-Dunn test
CDdiagram.Nemenyi	CD diagrams for the post-hoc Nemenyi test
classificationMetrics	Calculate some standard classification evaluation metrics of predictive performance
ComparisonResults	Class "ComparisonResults"
ComparisonResults-class	Class "ComparisonResults"
CV	Class "CV"
CV-class	Class "CV"
cvEstimates	Performance estimation using cross validation
EstCommon-class	Class '"EstCommon"'
EstimationMethod	Class "EstimationMethod"
EstimationMethod-class	Class "EstimationMethod"
EstimationResults	Class "EstimationResults"
EstimationResults-class	Class "EstimationResults"
estimationSummary	Obtain a set of descriptive statistics of the scores of a workflow on a task
EstimationTask	Class '"EstimationTask"'
EstimationTask-class	Class '"EstimationTask"'
getIterationsInfo	Obtaining the information returned by a workflow when applied to a task, on a particular iteration of the estimation process or on all iterations
getIterationsPreds	Obtaining the predictions returned by a workflow when applied to a task, on a particular iteration of the estimation process, or on all iterations
getScores	Obtaining the metric scores on the different iterations for a workflow / task combination
getWorkflow	Obtain the workflow object corresponding to an ID
hldEstimates	Performance estimation using holdout and random resampling
Holdout	Class "Holdout"
Holdout-class	Class "Holdout"
is.classification	Check if a certain predictive task is a classification problem
is.regression	Check if a certain predictive task is a regression problem
knnImp	Fill in NA values with the values of the nearest neighbours
LOOCV	Class "LOOCV"
LOOCV-class	Class "LOOCV"
loocvEstimates	Performance estimation using Leave One Out Cross Validation
mcEstimates	Performance estimation for time series prediction tasks using Monte Carlo
mergeEstimationRes	Merging several 'ComparisonResults' class objects
metricNames	The evaluation metrics estimated in an experiment
metricsSummary	Obtains a summary of the individual metric scores obtained by each workflow on a set of tasks.
MonteCarlo	Class "MonteCarlo"
MonteCarlo-class	Class "MonteCarlo"
pairedComparisons	Statistical hypothesis testing on the observed paired differences in estimated performance.
performanceEstimation	Estimate the predictive performance of modeling alternatives on different predictive tasks
plot-method	Class "ComparisonResults"
plot-method	Class "EstimationResults"
PredTask	Class "PredTask"
PredTask-class	Class "PredTask"
rankWorkflows	Provide a ranking of workflows involved in an estimation process.
regressionMetrics	Calculate some standard regression evaluation metrics of predictive performance
responseValues	Obtain the target variable values of a prediction task
results2table	Obtains a dplyr data frame table object containing all the results of an experiment
runWorkflow	Run a workflow on a predictive task
show-method	Class "Bootstrap"
show-method	Class "CV"
show-method	Class "ComparisonResults"
show-method	Class "EstimationResults"
show-method	Class '"EstimationTask"'
show-method	Class "Holdout"
show-method	Class "LOOCV"
show-method	Class "MonteCarlo"
show-method	Class "PredTask"
show-method	Class "Workflow"
signifDiffs	Obtains a list with the set of paired differences that are statistically significant according to a p-value threshold
smote	SMOTE algorithm for unbalanced classification problems
standardPOST	A function for applying post-processing steps to the predictions of a model
standardPRE	A function for applying data pre-processing steps
standardWF	A function implementing a standard workflow for prediction tasks
subset-method	Methods for Function 'subset' in Package 'performanceEstimation'
subset-methods	Methods for Function 'subset' in Package 'performanceEstimation'
summary-method	Class "ComparisonResults"
summary-method	Class "EstimationResults"
summary-method	Class "Workflow"
taskNames	The prediction tasks involved in an estimation experiment
timeseriesWF	A function implementing sliding and growing window standard workflows for time series forecasting tasks
topPerformer	Obtain the workflow that best performed in terms of a metric on a task
topPerformers	Obtain the best scores from a performance estimation experiment
Workflow	Class "Workflow"
Workflow-class	Class "Workflow"
workflowNames	The IDs of the workflows involved in an estimation experiment
workflowVariants	Generate (parameter) variants of a workflow