eval_report_pairs {clevr}

Evaluation Report for Linked Pairs

Description

Compute various evaluation measures for a set of predicted coreferent (linked) pairs given a set of ground truth coreferent pairs.

Usage

eval_report_pairs(true_pairs, pred_pairs, num_pairs = NULL, ordered = FALSE)

Arguments

true_pairs

set of true coreferent pairs stored in a matrix or data.frame, where rows index pairs and columns index the ids of the constituents. Any pairs not included are assumed to be non-coreferent. Duplicate pairs (including equivalent pairs with reversed ids) are automatically removed.

pred_pairs

set of predicted coreferent pairs, following the same specification as true_pairs.

num_pairs

the total number of coreferent and non-coreferent pairs, excluding equivalent pairs with reversed ids. If not provided, measures that depend on the number of true negatives will be returned as NA.

ordered

whether to treat the element pairs as ordered—i.e. whether pair (x, y) is distinct from pair (y, x) for x \neq y. Defaults to FALSE, which is appropriate for clustering, undirected link prediction, record linkage etc.

Value

Returns a list containing the following measures:

precision

see precision_pairs

recall

see recall_pairs

specificity

see specificity_pairs

sensitivity

see sensitivity_pairs

f1score

see f_measure_pairs

accuracy

see accuracy_pairs

balanced_accuracy

see balanced_accuracy_pairs

fowlkes_mallows

see fowlkes_mallows_pairs

See Also

The contingency_table_pairs function can be used to compute the contingency table for entity resolution or record linkage problems.

Examples

### Example where pairs/edges are undirected
# ground truth is 3-clique
true_pairs <- rbind(c(1,2), c(2,3), c(1,3))
# prediction misses one edge
pred_pairs <- rbind(c(1,2), c(2,3))
# total number of pairs assuming 3 elements
num_pairs <- 3 * (3 - 1) / 2
eval_report_pairs(true_pairs, pred_pairs, num_pairs)

### Example where pairs/edges are directed
# ground truth is a 3-star
true_pairs <- rbind(c(2,1), c(3,1), c(4,1))
# prediction gets direction of one edge incorrect
pred_pairs <- rbind(c(2,1), c(3,1), c(1,4))
# total number of pairs assuming 4 elements
num_pairs <- 4 * 4
eval_report_pairs(true_pairs, pred_pairs, num_pairs, ordered = TRUE)