oconnell {magree}

O'Connell-Dobson estimators for multiobserver agreement.

Description

Use the O'Connell-Dobson estimator of agreement for nominal or ordinal data. This includes a range of statistics on agreement for assuming either distinct or homogeneous items.

Usage

oconnell(X, weights=c("unweighted","linear","quadratic"), i=NULL, score = NULL)

Arguments

X

A matrix or data-frame with observations/subjects as rows and observers as columns.

weights

"unweighted"

For nominal categories - only perfect agreement is counted.

"linear"

For ordinal categories where disagreement is proportional to the distance between the categories. This is analogous to the agreement weights w_{i,j}=1-|score[i]-score[j]|/(max(score)-min(score)).

"quadratic"

For ordinal categories where disagreement is proportional to the square of the distance between the categories. This is analogous to the agreement weights w_{i,j}=1-(score[i]-score[j])^2/(max(score)-min(score))^2.

i

1. For nominal categories - only perfect agreement is counted.

2. For ordinal categories where disagreement is proportional to the distance between the categories. This is analogous to the agreement weights w_{i,j}=1-|score[i]-score[j]|/(max(score)-min(score)).

3. For ordinal categories where disagreement is proportional to the square of the distance between the categories. This is analogous to the agreement weights w_{i,j}=1-(score[i]-score[j])^2/(max(score)-min(score))^2.

This argument takes precedence over weights if it is specified.

score

The scores that are to be assigned to the categories. Currently, this defaults to 1:L, where L is the number of categories.

Details

The Fortran code from Professor Dianne O'Connell was adapted for R.

The output object is very similar to the Fortan code. Not all of the variance terms are currently used in the print, summary and plot methods.

Value

X

As input

i

As input

nrater

Number of observers

nscore

Number of categories

nsubj

Number of subjects

p1[j, k]

Probability of observer j giving score k when observers are distinct

p2[k]

Probability of score k when observers are homogeneous

w1[j, k]

Weighted average of d[] for observer j, score k

w2[k]

Weighted average of d[] for score k when observers are homogeneous

d[j]

Amount of disagreement for subject j

s1[j]

Chance-corrected agreement statistic for subject j when observers are distinct

s2[j]

Chance-corrected agreement statistic for subject j when observers are homogeneous; s[j]=1-d[j]/expdel.

delta[j, k]

j<k: amount of disagreement expected by change for observers j and k; j>k amount of disagreement expected by chance for observers j and k when observers are homogeneous

expd1

Amount of disagreement expected by chance in null case when observers are distinct

expd2

Amount of disagreement expected by chance when observers are homogeneous

dbar

Average value of d[] over all subjects

sav1

Chance-corrected agreement statistic over all subjects when observers are distinct

sav2

Chance-corrected agreement statistic over all subjects when observers are homogeneous

var0s1

Null variance of S when observers are distinct

var0s2

Null variance of S when observers are homogeneous

vars1

Unconstrained variance of S when observers are distinct

vars2

Unconstrained variance of S when observers are homogeneous

v0sav1

Null variance of Sav when observers are distinct

v0sav2

Null variance of Sav when observers are homogeneous

vsav1

Unconstrained variance of Sav when observers are distinct

vsav2

Unconstrained variance of Sav when observers are homogeneous

p0sav1

Probability of overall agreement due to chance when observers are distinct

p0sav2

Probability of overall agreement due to chance when observers are homogeneous

resp[i, j]

Response for observer i on subject j; transpose of X (BEWARE)

score(i)

Score associated with i'th category

call

As per sys.call(), to allow for using update

See Also

magree, schouten.

Examples

## Table 1 (O'Connell and Dobson, 1984)
summary(fit <- oconnell(landis, weights="unweighted"))
update(fit, weights="linear")
update(fit, weights="quadratic")

## Table 3 (O'Connell and Dobson, 1984)
slideTypeGroups <-
    list(c(2,3,5,26,31,34,42,58,59,67,70,81,103,120),
         c(7,10:13,17,23,30,41,51,55,56,60,65,71,73,76,86,87,105,111,116,119,124),
         c(4,6,24,25,27,29,39,48,68,77,79,94,101,102,117),
         c(9,32,36,44,52,62,84,95),
         c(35,53,69,72),
         c(8,15,18,19,47,64,82,93,98,99,107,110,112,115,121),
         c(1,16,22,49,63,66,78,90,100,113),
         c(28,37,40,61,108,114,118),
         106,
         43,
         83,
         c(54,57,88,91,126),
         c(74,104),
         38,
         46,
         c(89,122),
         c(80,92,96,123),
         85)
data.frame(SlideType=1:18,
           S1=sapply(slideTypeGroups,
                     function(ids) mean(fit$s1[as.character(ids)])),
           S2=sapply(slideTypeGroups,
                     function(ids) mean(fit$s2[as.character(ids)])))

## Table 5, O'Connell and Dobson (1984)
oconnell(landis==1)
oconnell(landis==2)
oconnell(landis==3)
oconnell(landis==4)
oconnell(landis==5)

## Plot of the marginal distributions
plot(fit)