fairness_accuracy_tradeoff {mlr3fairness} | R Documentation |
Plot Fairness Accuracy Trade-offs
Description
Provides visualization wrt. trade-offs between fairness and accuracy metrics across learners and resampling iterations. This can assist in gauging the optimal model from a set of options along with estimates of variance (through individual resampling iterations).
Usage
fairness_accuracy_tradeoff(object, ...)
Arguments
object |
(PredictionClassif | BenchmarkResult | ResampleResult)
|
... |
Arguments to be passed to methods. Such as:
|
Value
A 'ggplot2' object.
Protected Attributes
The protected attribute is specified as a col_role
in the corresponding Task()
:
<Task>$col_roles$pta = "name_of_attribute"
This also allows specifying more than one protected attribute,
in which case fairness will be considered on the level of intersecting groups defined by all columns
selected as a predicted attribute.
Examples
library("mlr3")
library("mlr3learners")
library("ggplot2")
# Setup the Fairness measure and tasks
task = tsk("adult_train")$filter(1:500)
learner = lrn("classif.ranger", predict_type = "prob")
fairness_measure = msr("fairness.tpr")
# Example 1 - A single prediction
learner$train(task)
predictions = learner$predict(task)
fairness_accuracy_tradeoff(predictions, fairness_measure, task = task)
# Example2 - A benchmark
design = benchmark_grid(
tasks = task,
learners = lrns(c("classif.featureless", "classif.rpart"),
predict_type = "prob", predict_sets = c("train", "test")),
resamplings = rsmps("cv", folds = 2)
)
bmr = benchmark(design)
fairness_accuracy_tradeoff(bmr, fairness_measure)