raters-package {raters}

A Modification of Fleiss' Kappa in case of Nominal and Ordinal Variables

Description

Computes a statistic as an index of inter-rater agreement among a set of raters in case of nominal or ordinal data.This procedure is based on a statistic not affected by Kappa paradoxes. In case of ordinal data, the weighted versions of the statistic has been developed using a matrix of linear or quadratic weights. The percentile Boostrap confidence interval is computed and the test argument allows to perform if the agreement is nil. The p value can be approximated using the Normal, Chi-squared distribution or using Monte Carlo algorithm in case of nominal data. Otherwise, the approximation and the Monte Carlo algorithm is computed. Fleiss' Kappa index is also shown in case of nominal data. In a nutshell, the function concordance can be used in case of nominal scale while the functions wlin.conc and wquad.conc can be used in case of ordinal data using linear or quadratic weights, respectively.

Details

Package:	raters
Type:	Package
Version:	2.0.3
License:	GPL(>=2)

Author(s)

Daniele Giardiello, Piero Quatto, Enrico Ripamonti and Stefano Vigliani

Maintainer: Daniele Giardiello <[e-mail]>

References

Fleiss, J.L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 378-382.

Falotico, R. Quatto, P. (2010). On avoiding paradoxes in assessing inter-rater agreement. Italian Journal of Applied Statistics 22, 151-160.

Falotico, R., Quatto, P. (2014). Fleiss' kappa statistic without paradoxes. Quality & Quantity, 1-8.

Marasini, D. Quatto, P. Ripamonti, E. (2014). Assessing the inter-rater agreement for ordinal data through weighted indexes. Statistical methods in medical research.

Examples

# Nominal data
data(diagnostic)
concordance(diagnostic,test="Normal")

# Ordinal data with linear weights
data(winetable)
set.seed(12345)
wlin.conc(winetable,test="MC")

# Ordinal data with quadratic weights
data(winetable)
set.seed(12345)
wquad.conc(winetable,test="MC")