bleu_corpus |
This function applies tokenization based on the 'tok' library and computes the BLEU score. An already initialized tokenizer can be provided using the 'tokenizer' argument or a valid huggingface identifier (string) can be passed. If the identifier is used only, the tokenizer is newly initialized on every call. |
bleu_corpus_ids |
Computes BLEU score (Papineni et al., 2002). |
bleu_sentence |
Compute BLEU for a Sentence with Tokenization |
bleu_sentence_ids |
Computes BLEU-Score (Papineni et al., 2002). |
validate_arguments |
Validate Arguments |
validate_references |
Validate References |