Estimators of divergence criteria for two normal distributions with Bayesian approach

Document Type : Original Scientific Paper

Authors

Department of Statistics‎, ‎Payame Noor University‎, ‎Tehran‎, ‎Iran

Abstract

The use of statistical distributions for modeling‎, ‎specifically evaluating the similarity between two probability distributions using various divergence measures‎, ‎has recently attracted the attention of many of researchers to measure in context of machine learning‎. ‎Given the importance of the topic‎, ‎this article introduces several the divergence criteria‎, ‎including Kullback-Leibler divergence‎, ‎total variation divergence‎, ‎alpha divergence‎, ‎and power divergence‎, ‎and computes the divergence parameters for two normal distributions‎. ‎The parameters are estimated using both maximum likelihood and Bayesian methods‎. ‎In the Bayesian approach‎, ‎a conjugate distribution is used as the prior‎, ‎taking into account the behavior of the parameters‎. ‎Finally‎, ‎the estimation methods for two normal distributions are evaluated bsased on the mean square error criterion.

Keywords

Main Subjects


Akaike, H. (1998). Information theory and an extension of the maximum likelihood principle. In Selected Papers of Hirotugu Akaike, pp. 199-213. New York: Springer.
Ding, R. and Mullhaupt, A. (2023). Empirical squared Hellinger distance estimator and generalizations to a family of α-divergence estimators. Entropy, 25(4):612.
Hellinger, E. (1909). Neue begründung der theorie quadratischer formen von unendlichvielen veränderlichen. Journal Für Die Reine Und Angewandte Mathematik, 1909(136):210–271.
Ji, S., Zhang, Z., Ying, S., Wang, L., Zhao, X. and Gao, Y. (2020). Kullback-Leibler divergence metric learning. IEEE Transactions on Cybernetics, 52(4):2047–2058.
Kaviani, M. (2025). Forecasting the return of government exchange-traded funds based on linear and nonlinear models in machine learning algorithms. Innovation Management and Operational Strategies, 6(1):59–69.
Koç, S., Erden, C., Ateş, Ç. and Ceviz, E. (2024). Evaluation of potential logistics village alternatives using bayesian best-worst method. Optimality, 1(1):100–120.
Kullback, S. and Leibler, R.A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22(1):79–86.
Moulin, P. and Veeravalli, V.V. (2019). Statistical Inference for Engineers and Data Scientists. Cambridge University Press.
Noh, Y.K., Sugiyama, M., Liu, S., du Plessis, M.C., Park, F.C. and Lee, D.D. (2018). Bias reduction and metric learning for nearest-neighbor estimation of Kullback-Leibler divergence. Neural Computation, 30(7):1930–1960.
Pardo, L. (2019). New developments in statistical information theory based on entropy and divergence measures. Entropy, 21(4):391.
Pérez-Cruz, F. (2008). Kullback-Leibler divergence estimation of continuous distributions. In 2008 IEEE International Symposium on Information Theory, pp. 1666-1670. IEEE.
Ugwu, D.N., Onyeagu, S.I. and Igbokwe, C.P. (2024). A new weighted T-X perks distribution: Characterization, simulation and applications. Optimality, 1(1):66–81.