check_new_rater {iotarelr} | R Documentation |
Check new rater
Description
Function for estimating the reliability of codings for a new rater based on Iota 2
Usage
check_new_rater(
true_values,
assigned_values,
con_step_size = 1e-04,
con_random_starts = 5,
con_max_iterations = 5000,
con_rel_convergence = 1e-12,
con_trace = FALSE,
fast = TRUE,
free_aem = FALSE
)
Arguments
true_values |
|
assigned_values |
|
con_step_size |
|
con_random_starts |
|
con_max_iterations |
|
con_rel_convergence |
|
con_trace |
|
fast |
|
free_aem |
|
Value
Returns a list
with the following three components:
The first component estimates_categorical_level
comprises all
elements that describe the ratings on a categorical level. The elements are
sub-divided into raw estimates and chance-corrected estimates.
raw_estimates
alpha_reliability:
A vector containing the Alpha Reliabilities for each category. These values represent probabilities.
beta_reliability:
A vector containing the Beta Reliabilities for each category. These values represent probabilities.
assignment_error_matrix:
An Assignment Error Matrix containing the conditional probabilities for assigning a unit of category i to categories 1 to n.
iota:
A vector containing the Iota values for each category.
elements_chance_corrected
alpha_reliability:
A vector containing the chance-corrected Alpha Reliabilities for each category.
beta_reliability:
A vector containing the chance-corrected Beta Reliabilities for each category.
The second component estimates_scale_level
contains elements to
describe the quality of the ratings on a scale level. It contains the
following elements:
iota_index:
The Iota Index representing the reliability on a scale level.
iota_index_d4:
The Static Iota Index, which is a transformation of the original Iota Index, in order to consider the uncertainty of estimation.
iota_index_dyn2:
The Dynamic Iota Index, which is a transformation of the original Iota Index, in order to consider the uncertainty of estimation.
The third component information
contains important information
regarding the parameter estimation. It comprises the following elements:
log_likelihood:
Log-likelihood of the best solution.
convergence:
If estimation converged 0, otherwise 1.
est_true_cat_sizes:
Estimated categorical sizes. This is the estimated amount of the categories.
conformity:
0
if the solution is in line with assumptions of weak superiority. A number greater 0 indicates the number of violations of the assumption of weak superiority.random_starts:
Numer of random starts for the EM algorithm.
boundaries:
False
if the best solution does not contain boundary values.True
if the best solution does contain boundary valuesp_boundaries:
Percentage of solutions with boundary values during estimation.
call:
Name of the function that created the object.
n_rater:
Number of raters.
n_cunits:
Number of coding units.
Note
The returned object contains further slots since the returned object is
of class iotarelr_iota2
. These slots are empty because they are not part of the
estimation within this function.
Please do not use the measures on the scale level if the Assignment Error Matrix was freely estimated since this kind of matrix is not conceptualized for comparing the coding process with random guessing.
References
Florian Berding and Julia Pargmann (2022). Iota Reliability Concept of the Second Generation. Measures for Content Analysis Done by Humans or Artificial Intelligences. Berlin:Logos. https://doi.org/10.30819/5581