q_reliability {AATtools}R Documentation

Compute psychological experiment reliability


This function can be used to compute an exact reliability score for a psychological task whose results involve a difference score. The resulting intraclass correlation coefficient is equivalent to the average all possible split-half reliability scores. It ranges from -1 to 1, with -1 implying that all variance in the data is explained by within-subjects variability, 1 implying that all variance is explained by between-subjects variability, and 0 implying that within-subjects and between-subjects variability contribute equally to the total variance in the sample.


q_reliability(ds, subjvar, formula, aatterm = NA)

q_reliability2(ds, subjvar, splitvars, rtvar, dscore = F, na.rm = F)

## S3 method for class 'qreliability'
print(x, ...)

## S3 method for class 'qreliability'
plot(x, ...)



a long-format data.frame


name of the subject variable


a formula predicting the participant's reaction time using trial-level variables such as movement direction and stimulus category


a string denoting the term in the formula that contains the participant's approach bias


Vector of column names over which to split the data to compute difference scores. This can be used to compute the reliability of single, double, or even triple difference scores.


Column name of the variable containing reaction times


If true, reliability will be computed for a difference score that is divided by the subject's standard deviation (as in D-scores)


If true, remove rows with missing values from the data


a qreliability object


Other arguments passed to the generic print and plot functions.


a qreliability object, containing the reliability coefficient, and a data.frame with participants' bias scores and score variance.

Please note that the valence of the bias scores may or may not correspond with approach and avoidance. If you plan to use these scores in your analyses, always verify that they are in the right direction by correlating them with independently calculated bias scores, for example using aat_compute().


Sercan Kahveci


# Double-difference score reliability
              formula= RT ~ is_pull * is_target, aatterm = "is_pull:is_target")

# Single-difference reliability for target stimuli
q_reliability(ds=erotica[erotica$is_target ==1,],subjvar="subject",
              formula= RT ~ is_pull, aatterm = "is_pull")

# Reliability of the mean reaction time of approaching target stimuli (no difference score)
q_reliability(ds=erotica[erotica$is_target ==1 & erotica$is_pull ==1,],subjvar="subject",
              formula= RT ~ 1, aatterm = "1")

              splitvars=c("is_pull", "is_target"),rtvar="RT")

[Package AATtools version 0.0.2 Index]