APPEstimation-package {APPEstimation}

R function to calculate model performance measure adjusted for predictor distributions.

Description

This package provides the function to estimate model performance measures (L_1, L_2, C-statistics). The difference in the distribution of predictors between two datasets (training and validation) is adjusted by a density ratio estimate.

Details

Package:	APPEstimation
Type:	Package
Title:	Adjusted Prediction Model Performance Estimation
Version:	0.1.1
Depends:	densratio
Date:	2018-1-4
Author:	Eisuke Inoue, Hajime Uno
Maintainer:	Eisuke Inoue <[e-mail]>
Description:	Calculating predictive model performance measures adjusted for predictor distributions using density ratio method (Sugiyama et al., (2012, ISBN:9781139035613)). L1 and L2 error for continuous outcome and C-statistics for binomial outcome are computed.
License:	GPL-2

Index of help topics:

APPEstimation-package   R function to calculate model performance
                        measure adjusted for predictor distributions.
appe.glm                C-statistics adjusted for predictor
                        distributions
appe.lm                 L_1 and L_2 errors adjusted for predictor
                        distributions
cvalest.bin             Estimation of C-statistics
densratio.appe          A wrapper function

Author(s)

Eisuke Inoue, Hajime Uno

Maintainer: Eisuke Inoue <[e-mail]>

References

Sugiyama, M., Suzuki, T. & Kanamori, T. Density Ratio Estimation in Machine Learning. Cambridge University Press 2012. ISBN:9781139035613.

Examples

set.seed(100)

# generating learning data
n0  = 100
Z   = cbind(rbeta(n0, 5, 5), rbeta(n0, 5, 5))
Y   = apply(Z, 1, function (xx) {
        rbinom(1, 1, (1/(1+exp(-(sum(c(-2,2,2) * c(1,xx)))))))})
dat = data.frame(Y=Y, Za=Z[,1], Zb=Z[,2])

# the model to be evaluated
mdl = glm(Y~., binomial, data=dat)

# validation dataset, with different centers on predictors
n1   = 100
Z1   = cbind(rbeta(n1, 6, 4), rbeta(n1, 6, 4))
Y1   = apply(Z1, 1, function (xx) {
         rbinom(1, 1, (1/(1+exp(-(sum(c(-2,2,2) * c(1,xx)))))))})
dat1 = data.frame(Y=Y1, Za=Z1[,1], Zb=Z1[,2])

# calculation of L1 and L2 for this model
appe.glm(mdl, dat, dat1, reps=0)