network-scores {bnlearn}

Network scores

Description

Overview of the network scores implemented in bnlearn, with the respective reference publications.

Details

Available scores (and the respective labels) for discrete Bayesian networks (categorical variables) are:

• the multinomial log-likelihood (loglik) score, which is equivalent to the entropy measure used in Weka.

• the Akaike Information Criterion (AIC) score (aic).

• the Bayesian Information Criterion (BIC) score (bic), which is equivalent to the Minimum Description Length (MDL) and is also known as Schwarz Information Criterion.

  Chickering DM (1995). "A Transformational Characterization of Equivalent Bayesian Network Structures". Proceedings of the Eleventh Annual Conference on Uncertainty in Artificial Intelligence, 87–98.

• the extended Bayesian Information Criterion (ebic), which adds a second penalty to BIC to penalize dense networks.

  Foygel R, Drton M (2010). "Extended Bayesian Information Criteria for Gaussian Graphical Models". NIPS 23, 604–612.

• the predictive log-likelihood (pred-loglik) computed on a separate test set.

  Chickering DM, Heckerman D (2000). "A Comparison of Scientific and Engineering Criteria for Bayesian Model Selection". Statistics and Computing, 10:55–62.

  Scutari M, Vitolo C, Tucker A (2019). "Learning Bayesian Networks from Big Data with Greedy Search: Computational Complexity and Efficient Implementation". Statistics and Computing, 25(9):1095–1108.

• the logarithm of the Bayesian Dirichlet equivalent (uniform) score (bde) (also denoted BDeu), a score equivalent Dirichlet posterior density.

  Heckerman D, Geiger D, Chickering DM (1995). "Learning Bayesian Networks: The Combination of Knowledge and Statistical Data". Machine Learning, 20(3):197–243.

  Castelo R, Siebes A (2000). "Priors on Network Structures. Biasing the Search for Bayesian Networks". International Journal of Approximate Reasoning, 24(1):39–57.

• the logarithm of the Bayesian Dirichlet sparse score (bds) (BDs), a sparsity-inducing Dirichlet posterior density (not score equivalent).

  Scutari M (2016). "An Empirical-Bayes Score for Discrete Bayesian Networks". Journal of Machine Learning Research, 52:438–448.

• the logarithm of the Bayesian Dirichlet score with Jeffrey's prior (not score equivalent).

  Suzuki J (2016). "A Theoretical Analysis of the BDeu Scores in Bayesian Network Structure Learning". Behaviormetrika, 44(1):97–116.

• the logarithm of the modified Bayesian Dirichlet equivalent score (mbde) for mixtures of experimental and observational data (not score equivalent).

  Cooper GF, Yoo C (1999). "Causal Discovery from a Mixture of Experimental and Observational Data". Proceedings of the Fifteenth Annual Conference on Uncertainty in Artificial Intelligence, 116–125.

• the logarithm of the locally averaged Bayesian Dirichlet score (bdla, not score equivalent).

  Cano A, Gomez-Olmedo M, Masegosa AR, Moral S (2013). "Locally Averaged Bayesian Dirichlet Metrics for Learning the Structure and the Parameters of Bayesian Networks". International Journal of Approximate Reasoning, 54:526–540.

• the logarithm of the K2 score (k2), a Dirichlet posterior density (not score equivalent).

  Korb K, Nicholson AE (2010). Bayesian Artificial Intelligence. Chapman & Hall/CRC, 2nd edition.

• the logarithm of the factorized normalized maximum likelihood score (fnml, not score equivalent).

  Silander T, Roos T, Kontkanen P, Myllymaki P (2008). "Factorized Normalized Maximum Likelihood Criterion for Learning Bayesian Network Structures". Proceedings of the 4th European Workshop on Probabilistic Graphical Models, 257–272.

• the logarithm of the quotient normalized maximum likelihood (qnml).

  Silander T, Leppa-Abo J, Jaasaari, Roos T (2018). "Quotient Normalized Maximum Likelihood Criterion for Learning Bayesian Network Structures". Proceedings of Machine Learning Research, 84:948–957.

• the node-average (log-)likelihood (nal) and the penalized node-average (log-)likelihood (pnal).

  Bodewes T, Scutari M (2021). "Learning Bayesian Networks from Incomplete Data with the Node-Averaged Likelihood". International Journal of Approximate Reasoning, 138:145–160.

Available scores (and the respective labels) for Gaussian Bayesian networks (normal variables) are:

• the multivariate Gaussian log-likelihood (loglik-g) score.

• the corresponding Akaike Information Criterion (AIC) score (aic-g).

• the corresponding Bayesian Information Criterion (BIC) score (bic-g).

  Geiger D, Heckerman D (1994). "Learning Gaussian Networks". Proceedings of the Tenth Annual Conference on Uncertainty in Artificial Intelligence, 235–243.

• the extended Bayesian Information Criterion (ebic-g), which adds a second penalty to BIC to penalize dense networks.

  Foygel R, Drton M (2010). "Extended Bayesian Information Criteria for Gaussian Graphical Models". NIPS 23, 604–612.

• the predictive log-likelihood (pred-loglik-g) computed on a separate test set. The reference paper is the same as that for pred-loglik. It is currently implemented to be score-equivalent like pred-loglik, but that may be subject to change.

• a score equivalent Gaussian posterior density (bge).

  Kuipers J, Moffa G, Heckerman D (2014). "Addendum on the Scoring of Gaussian Directed Acyclic Graphical Models". The Annals of Statistics, 42(4):1689–1691.

• the node-average (log-)likelihood (nal-g) and the penalized node-average (log-)likelihood (pnal-g).

  Bodewes T, Scutari M (2021). "Learning Bayesian Networks from Incomplete Data with the Node-Averaged Likelihood". International Journal of Approximate Reasoning, 138:145–160.

Available scores (and the respective labels) for hybrid Bayesian networks (mixed categorical and normal variables) are:

• the conditional linear Gaussian log-likelihood (loglik-cg) score.

• the corresponding Akaike Information Criterion (AIC) score (aic-cg).

• the corresponding Bayesian Information Criterion (BIC) score (bic-cg).

• the extended Bayesian Information Criterion (ebic-cg), which adds a second penalty to BIC to penalize dense networks.

  Foygel R, Drton M (2010). "Extended Bayesian Information Criteria for Gaussian Graphical Models". NIPS 23, 604–612.

• the predictive log-likelihood (pred-loglik-cg) computed on a separate test set. The reference paper is the same as that for pred-loglik.

• the node-average (log-)likelihood (nal-cg) and the penalized node-average (log-)likelihood (pnal-cg).

  Bodewes T, Scutari M (2021). "Learning Bayesian Networks from Incomplete Data with the Node-Averaged Likelihood". International Journal of Approximate Reasoning, 138:145–160.

Other scores (and the respective labels):

• a custom decomposable (custom) score interface that takes an R function as an argument. It can be used to trial experimental score functions without having to code them in C and hook them up to the internals of bnlearn.