Statistical inference for Kullback–Leibler and total variation divergences for two normal populations with a common unknown variance

Document Type : Original Scientific Paper

Authors

Department of Statistics‎, ‎Payame Noor University‎, ‎Tehran‎, ‎Iran

10.22034/jsmta.2026.24790.1224

Abstract

This article focuses on the estimation of two divergence measures, the Kullback–Leibler divergence and the total variation divergence, between two normal distributions with a common unknown variance. We provide the maximum likelihood estimator and a Bayesian estimator for these divergences. In the Bayesian framework, improper prior distributions are considered for the population parameters, and both Bayesian point estimators and interval estimators, including credible intervals and highest posterior density intervals, are constructed. In the classical approach, we also present the asymptotic normal confidence intervals. The performance of the point estimators is evaluated using the root mean squared error criterion, while the interval estimators are assessed based on coverage probability and average length through a simulation study. Finally, we use a real dataset to illustrate the proposed methods.

Keywords

Main Subjects


Afshar Moghaddam, S., Nasiri, P. and Yarmohammadi, M. (2025). Estimators of divergence criteria for two normal distributions with Bayesian approach. Journal of Statistical Modelling: Theory and Applications, 6(1):83–91.
Akaike, H. (1998). Information theory and an extension of the maximum likelihood principle. In Selected Papers of Hirotugu Akaike, pp. 199–213, New York, NY: Springer New York.
Box, G.E.P. and Tiao, G.C. (1973). Bayesian Inference in Statistical Analysis. Addison-Wesley Publishing Company.
Chen, M.H. and Shao, Q.M. (1999). Monte Carlo estimation of Bayesian credible and HPD intervals. Journal of Computational and Graphical Statistics, 8(1):69–92.
Ji, S., Zhang, Z., Ying, S., Wang, L., Zhao, X. and Gao, Y. (2020). Kullback-Leibler divergence metric learning. IEEE Transactions on Cybernetics, 52(4):2047–2058.
Kullback, S. and Leibler, R.A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22(1):79–86.
Moulin, P. and Veeravalli, V.V. (2018). Statistical Inference for Engineers and Data Scientists. Cambridge University Press.
Noh, Y.K., Sugiyama, M., Liu, S., Plessis, M.C., Park, F.C. and Lee, D.D. (2014). Bias reduction and metric learning for nearest-neighbor estimation of Kullback-Leibler divergence. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, pp. 669–677. PMLR, Iceland.
Pardo, L. (2019). New developments in statistical information theory based on entropy and divergence measures. Entropy, 21(4):391.
Pérez-Cruz, F. (2008). Kullback-Leibler divergence estimation of continuous distributions. In 2008 IEEE International Symposium on Information Theory, 1666–1670.
Weerahandi, S. (1993). Generalized confidence intervals. Journal of the American Statistical Association, 88(423):899–905.
Weerahandi, S. (1995). Exact Statistical Methods for Data Analysis. New York: Springer Verlag.
Xia, Z.P., Yu, J.Y., Cheng, L.D., Liu, L.F. and Wang, W.M. (2009). Study on the breaking strength of jute fibers using modified Weibull distribution. Journal of Composites Part A: Applied Science and Manufacturing, 40(1):54–59.

Articles in Press, Corrected Proof
Available Online from 22 November 2026
  • Receive Date: 20 June 2026
  • Revise Date: 17 October 2026
  • Accept Date: 08 November 2026
  • First Publish Date: 22 November 2026
  • Publish Date: 22 November 2026