ceteris_paribus_cutoff {fairmodels} | R Documentation |
Ceteris paribus cutoff
Description
Ceteris paribus cutoff is way to check how will parity loss behave if only cutoff for one subgroup was changed.
By using parameter new_cutoffs
parity loss for metrics with new cutoffs will be calculated. Note that cutoff for subgroup (passed as parameter) will
change no matter new_cutoff
's value at that position. When parameter cumulated
is set to true, all metrics will be summed and facets will
collapse to one plot with different models on it. Sometimes due to the fact that some metric might contain NA for all cutoff values, cumulated plot might be present without
this model.
Usage
ceteris_paribus_cutoff(
x,
subgroup,
new_cutoffs = NULL,
fairness_metrics = c("ACC", "TPR", "PPV", "FPR", "STP"),
grid_points = 101,
cumulated = FALSE
)
Arguments
x |
object of class |
subgroup |
character, name of subgroup (level in protected variable) |
new_cutoffs |
list of cutoffs with names matching those of subgroups. Each value should represent cutoff for particular subgroup. Position corresponding to subgroups in levels will be changed. Default is NULL |
fairness_metrics |
character, name of parity_loss metric or vector of multiple metrics, for full metric names check |
grid_points |
numeric, grid for cutoffs to test. Number of points between 0 and 1 spread evenly. |
cumulated |
logical, if |
Value
ceteris_paribus_cutoff
data.frame
containing information about label, metric and parity_loss at particular cutoff
Examples
data("compas")
# positive outcome - not being recidivist
two_yr_recidivism <- factor(compas$Two_yr_Recidivism, levels = c(1, 0))
y_numeric <- as.numeric(two_yr_recidivism) - 1
compas$Two_yr_Recidivism <- two_yr_recidivism
lm_model <- glm(Two_yr_Recidivism ~ .,
data = compas,
family = binomial(link = "logit")
)
explainer_lm <- DALEX::explain(lm_model, data = compas[, -1], y = y_numeric)
fobject <- fairness_check(explainer_lm,
protected = compas$Ethnicity,
privileged = "Caucasian"
)
cpc <- ceteris_paribus_cutoff(fobject, "African_American")
plot(cpc)
rf_model <- ranger::ranger(Two_yr_Recidivism ~ .,
data = compas,
probability = TRUE,
num.trees = 200
)
explainer_rf <- DALEX::explain(rf_model, data = compas[, -1], y = y_numeric)
fobject <- fairness_check(explainer_lm, explainer_rf,
protected = compas$Ethnicity,
privileged = "Caucasian"
)
cpc <- ceteris_paribus_cutoff(fobject, "African_American")
plot(cpc)