kullback_leibler_disc {oHMMed}

Calculate a Kullback-Leibler Divergence for a Discrete Distribution

Description

Calculate a Kullback-Leibler Divergence for a Discrete Distribution

Usage

kullback_leibler_disc(p, q)

Arguments

p

(numeric) probabilities

q

(numeric) probabilities

Details

The Kullback-Leibler divergence for a discrete distribution is calculated as follows:

\sum_{i=1}^n p_i \log\Big(\frac{p_i}{q_i}\Big)

Value

Numeric vector

Examples

# Simulate n Poisson distributed variates
n <- 1000
dist1 <- rpois(n, lambda = 1)
dist2 <- rpois(n, lambda = 5)
dist3 <- rpois(n, lambda = 20)

# Generate common factor levels
x_max <- max(c(dist1, dist2, dist3))
all_levels <- 0:x_max

# Estimate probability mass functions 
pmf_dist1 <- table(factor(dist1, levels = all_levels)) / n
pmf_dist2 <- table(factor(dist2, levels = all_levels)) / n
pmf_dist3 <- table(factor(dist3, levels = all_levels)) / n

# Visualise PMFs
barplot(pmf_dist1, col = "green", xlim = c(0, x_max))
barplot(pmf_dist2, col = "red", add = TRUE)
barplot(pmf_dist3, col = "blue", add = TRUE)

# Calculate distances
kullback_leibler_disc(pmf_dist1, pmf_dist2)
kullback_leibler_disc(pmf_dist1, pmf_dist3)
kullback_leibler_disc(pmf_dist2, pmf_dist3)