f_measure_pairs {clevr}

F-measure of Linked Pairs

Description

Computes the F-measure (a.k.a. F-score) of a set of predicted coreferent (linked) pairs given a set of ground truth coreferent pairs.

Usage

f_measure_pairs(true_pairs, pred_pairs, beta = 1, ordered = FALSE)

Arguments

true_pairs

set of true coreferent pairs stored in a matrix or data.frame, where rows index pairs and columns index the ids of the constituents. Any pairs not included are assumed to be non-coreferent. Duplicate pairs (including equivalent pairs with reversed ids) are automatically removed.

pred_pairs

set of predicted coreferent pairs, following the same specification as true_pairs.

beta

non-negative weight. A value of 0 assigns no weight to recall (i.e. the measure reduces to precision), while larger values assign increasing weight to recall. A value of 1 weights precision and recall equally.

ordered

whether to treat the element pairs as ordered—i.e. whether pair (x, y) is distinct from pair (y, x) for x \neq y. Defaults to FALSE, which is appropriate for clustering, undirected link prediction, record linkage etc.

Details

The \beta-weighted F-measure is defined as the weighted harmonic mean of precision P and recall R:

(1 + \beta^2)\frac{P \cdot R}{\beta^2 \cdot P + R}.

References

Van Rijsbergen, C. J. "Information Retrieval." (2nd ed.). Butterworth-Heinemann, USA, (1979).

Examples

true_pairs <- rbind(c(1,2), c(2,3), c(1,3)) # ground truth is 3-clique
pred_pairs <- rbind(c(1,2), c(2,3))         # prediction misses one edge
num_pairs <- 3                              # assuming 3 elements
f_measure_pairs(true_pairs, pred_pairs, num_pairs)