get_coder_metrics {aifeducation}

Calculate reliability measures based on content analysis

Description

This function calculates different reliability measures which are based on the empirical research method of content analysis.

Usage

get_coder_metrics(
  true_values = NULL,
  predicted_values = NULL,
  return_names_only = FALSE
)

Arguments

true_values

factor containing the true labels/categories.

predicted_values

factor containing the predicted labels/categories.

return_names_only

bool If TRUE returns only the names of the resulting vector. Use FALSE to request computation of the values.

Value

If return_names_only=FALSE returns a vector with the following reliability measures: #'

• iota_index: Iota Index from the Iota Reliability Concept Version 2.

• min_iota2: Minimal Iota from Iota Reliability Concept Version 2.

• avg_iota2: Average Iota from Iota Reliability Concept Version 2.

• max_iota2: Maximum Iota from Iota Reliability Concept Version 2.

• min_alpha: Minmal Alpha Reliability from Iota Reliability Concept Version 2.

• avg_alpha: Average Alpha Reliability from Iota Reliability Concept Version 2.

• max_alpha: Maximum Alpha Reliability from Iota Reliability Concept Version 2.

• static_iota_index: Static Iota Index from Iota Reliability Concept Version 2.

• dynamic_iota_index: Dynamic Iota Index Iota Reliability Concept Version 2.

• kalpha_nominal: Krippendorff's Alpha for nominal variables.

• kalpha_ordinal: Krippendorff's Alpha for ordinal variables.

• kendall: Kendall's coefficient of concordance W.

• kappa2_unweighted: Cohen's Kappa unweighted.

• kappa2_equal_weighted: Weighted Cohen's Kappa with equal weights.

• kappa2_squared_weighted: Weighted Cohen's Kappa with squared weights.

• kappa_fleiss: Fleiss' Kappa for multiple raters without exact estimation.

• percentage_agreement: Percentage Agreement.

• balanced_accuracy: Average accuracy within each class.

• gwet_ac: Gwet's AC1/AC2 agreement coefficient.

If return_names_only=TRUE returns only the names of the vector elements.

See Also

Other Auxiliary Functions: array_to_matrix(), calc_standard_classification_measures(), check_embedding_models(), clean_pytorch_log_transformers(), create_iota2_mean_object(), create_synthetic_units(), generate_id(), get_folds(), get_n_chunks(), get_stratified_train_test_split(), get_synthetic_cases(), get_train_test_split(), is.null_or_na(), matrix_to_array_c(), split_labeled_unlabeled(), summarize_tracked_sustainability(), to_categorical_c()