multinomial_naive_bayes {naivebayes}

Multinomial Naive Bayes Classifier

Description

multinomial_naive_bayes is used to fit the Multinomial Naive Bayes model.

Usage

multinomial_naive_bayes(x, y, prior = NULL, laplace = 0.5, ...)

Arguments

x

numeric matrix with integer predictors (matrix or dgCMatrix from Matrix package).

y

class vector (character/factor/logical).

prior

vector with prior probabilities of the classes. If unspecified, the class proportions for the training set are used. If present, the probabilities should be specified in the order of the factor levels.

laplace

value used for Laplace smoothing (additive smoothing). Defaults to 0.5.

...

not used.

Details

This is a specialized version of the Naive Bayes classifier, where the features represent frequencies generated by a multinomial distribution.

Sparse matrices of class "dgCMatrix" (Matrix package) are supported in order to speed up calculation times.

Please note that the Multinomial Naive Bayes is not available through the naive_bayes function.

Value

multinomial_naive_bayes returns an object of class "multinomial_naive_bayes" which is a list with following components:

data

list with two components: x (matrix with predictors) and y (class variable).

levels

character vector with values of the class variable.

laplace

amount of Laplace smoothing (additive smoothing).

params

matrix with class conditional parameter estimates.

prior

numeric vector with prior probabilities.

call

the call that produced this object.

Author(s)

Michal Majka, [e-mail]

References

Manning, C.D., Raghavan, P., & Schütze, H. (2008). An Introduction to Information Retrieval. Cambridge: Cambridge University Press (Chapter 13). Available at https://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf

See Also

predict.multinomial_naive_bayes, tables, get_cond_dist, %class%, coef.multinomial_naive_bayes

Examples

# library(naivebayes)

### Simulate the data:
set.seed(1)
cols <- 3 # words
rows <- 10000 # all documents
rows_spam <- 100 # spam documents

prob_word_non_spam <- prop.table(runif(cols))
prob_word_spam <- prop.table(runif(cols))

M1 <- t(rmultinom(rows_spam, size = cols, prob = prob_word_spam))
M2 <- t(rmultinom(rows - rows_spam, size = cols, prob = prob_word_non_spam))
M <- rbind(M1, M2)
colnames(M) <- paste0("word", 1:cols) ; rownames(M) <- paste0("doc", 1:rows)
head(M)
y <- c(rep("spam", rows_spam), rep("non-spam", rows - rows_spam))

### Train the Multinomial Naive Bayes
laplace <- 1
mnb <- multinomial_naive_bayes(x = M, y = y, laplace = laplace)
summary(mnb)

# Classification
head(predict(mnb, newdata = M, type = "class")) # head(mnb %class% M)

# Posterior probabilities
head(predict(mnb, newdata = M, type = "prob")) # head(mnb %prob% M)

# Parameter estimates
coef(mnb)

# Compare
round(cbind(non_spam = prob_word_non_spam, spam = prob_word_spam), 3)



### Sparse data: train the Multinomial Naive Bayes
library(Matrix)
M_sparse <- Matrix(M, sparse = TRUE)
class(M_sparse) # dgCMatrix

# Fit the model with sparse data
mnb_sparse <- multinomial_naive_bayes(M_sparse, y, laplace = laplace)

# Classification
head(predict(mnb_sparse, newdata = M_sparse, type = "class"))

# Posterior probabilities
head(predict(mnb_sparse, newdata = M_sparse, type = "prob"))

# Parameter estimates
coef(mnb_sparse)