entropyData {abn} | R Documentation |
This function empirically estimates the Shannon entropy from a table of counts using the observed frequencies.
entropyData(freqs.table)
freqs.table |
a table of counts. |
The general concept of entropy is defined for probability distributions.
The entropyData()
function estimates empirical entropy from data.
The probability is estimated from data using frequency tables.
Then the estimates are plug-in in the definition of the entropy to return
the so-called empirical entropy. A common known problem of empirical entropy
is that the estimations are biased due to the sampling noise.
It is also known that the bias will decrease as the sample size increases.
Shannon's entropy estimate on natural logarithm scale.
Cover, Thomas M, and Joy A Thomas. (2012). "Elements of Information Theory". John Wiley & Sons.
## Generate random variable
rv <- rnorm(n = 100, mean = 5, sd = 2)
dist <- list("gaussian")
names(dist) <- c("rv")
## Compute the entropy through discretization
entropyData(freqs.table = discretization(data.df = rv, data.dists = dist,
discretization.method = "sturges", nb.states = FALSE))