pa_evaluate {predicts}

Presence/absence Model evaluation

Description

Evaluation of models with presence/absence data. Given a vector of presence and a vector of absence values, confusion matrices are computed for a sequence of thresholds, and model evaluation statistics are computed for each confusion matrix / threshold.

Usage

pa_evaluate(p, a, model=NULL, x=NULL, tr, ...)

Arguments

p

either (1) predictions for presence points (model and x are not NULL); or (2) predictor values for presence points (model is not NULL, x is NULL; or locations for presence points (model and x are not NULL)

a

as above for absence or background points

model

A fitted model used to make predictions

x

SpatRaster used to extract predictor values from

tr

Optional. a vector of threshold values to use for computing the confusion matrices

...

Additional arguments passed on to predict(model,...)

Value

pa_ModelEvaluation object

details

A pa_ModelEvaluation object has the the following slots

presence:

presence values used

absence:

absence values used

confusion:

confusion matrix for each threshold

stats:

statistics that are not threshold dependent

tr_stats:

statistics that are threshold dependent

thresholds:

optimal thresholds to classify values into presence and absence

stats has the following values

np:

number of presence points

na:

number of absence points

auc:

Area under the receiver operator (ROC) curve

pauc:

p-value for the AUC (for the Wilcoxon test W statistic

cor:

Correlation coefficient

pcor:

p-value for correlation coefficient

prevalence:

Prevalence

ODP:

Overall diagnostic power

tr_stats has the following values

tresholds:

vector of thresholds used to compute confusion matrices

CCR:

Correct classification rate

TPR:

True positive rate

TNR:

True negative rate

FPR:

False positive rate

FNR:

False negative rate

PPP:

Positive predictive power

NPP:

Negative predictive power

MCR:

Misclassification rate

OR:

Odds-ratio

kappa:

Cohen's kappa

thresholds has the following values

max_kappa:

the threshold at which kappa is highest

max_spec_sens:

the threshold at which the sum of the sensitivity (true positive rate) and specificity (true negative rate) is highest

no_omission:

the highest threshold at which there is no omission

prevalence:

modeled prevalence is closest to observed prevalence

equal_sens_spec:

equal sensitivity and specificity

References

Fielding, A.H. and J.F. Bell, 1997. A review of methods for the assessment of prediction errors in conservation presence/absence models. Environmental Conservation 24:38-49

Liu, C., M. White & G. Newell, 2011. Measuring and comparing the accuracy of species distribution models with presence-absence data. Ecography 34: 232-243.

Examples

set.seed(0)
# p has the predicted values for 50 known cases (locations) 
# with presence of the phenomenon (species)
p <- rnorm(50, mean=0.6, sd=0.3)
# a has the predicted values for 50 background locations (or absence)
a <- rnorm(50, mean=0.4, sd=0.4)

e <- pa_evaluate(p=p, a=a)
e

e@stats

plot(e, "ROC")
plot(e, "TPR")
plot(e, "boxplot")
plot(e, "density")

str(e)