Mixtures of the normal mean-variance of Lindley factor analysis model with missing data

Document Type : Original Scientific Paper

Authors

1 Department of Statistics‎, ‎Yazd University‎, Yazd,‎ ‎Iran

2 Department of Statistics‎, ‎University of Kashan‎, ‎Kashan

Abstract

For a heterogeneous community comprised of multiple sub-communities‎, ‎the model-based clustering method stands out as a suitable recommendation‎. ‎Moreover‎, ‎incomplete data collection and information loss may occur due to a variety of causes‎. ‎In this paper‎, ‎our focus is on investigating the mixture of factor analysis model in the presence of missing data‎. ‎Here‎, ‎the latent factors and errors within each sub-cluster exhibit non-normal characteristics and adhere to the normal mean-variance mixture of Lindley distribution‎. ‎This model is termed the mixture of normal mean-variance mixture of Lindley factor analysis‎. ‎To estimate model parameters and generate a single imputation of potential missing values under the missing with random mechanism‎, ‎we introduce a generalized expectation-maximization algorithm‎. ‎The number of factors and mixture components are determined by the evaluation criteria of the model‎. ‎The proposed model's advantage is validated through a real dataset and simulation studies‎, ‎demonstrating its superior performance compared to existing models.

Keywords

Main Subjects

References

Aitken, A. (1925). On Bernoulli's numerical solution of algebraic equations. Proceedings of the Royal Society of Edinburgh, 46:289–305.
Arslan, O. (2010). An alternative multivariate skew Laplace distribution: Properties and estimation. Statistical Papers, 51(4):865–887.
Azzalini, A. (2005). The skew-normal distribution and related multivariate families. Scandinavian Journal of Statistics, 32(2):159–188.
Azzalini, A. and Capitanio, A. (2003). Distributions generated by perturbation of symmetry with emphasis on a multivariate skew t-distribution. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65(2):367–389.
Bai, J. and Li, K. (2012). Statistical analysis of factor models of high dimension. The Annals of Statistics, 40(1):436–465.
Barndorff-Nielsen, O. (1977). Exponentially decreasing distributions for the logarithm of particle size. Proceedings of the Royal Society of London A Mathematical and Physical Sciences, 353(1674):401–419.
Basilevsky, A.T. (2009). Statistical Factor Analysis and Related Methods: Theory and Applications. John Wiley & Sons.
Biernacki, C., Celeux, G. and Govaert, G. (2000). Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(7):719–725.
Darijani, M., Zakerzadeh, H. and Jafari, A.A. (2024). Introducing a family of distributions by using the class of normal mean-variance mixture. Journal of Statistical Theory and Practice, 18(1):17.
Dempster, A.P., Laird, N.M. and Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society Series B: Statistical Methodology, 39(1):1–38.
De Roover, K., Vermunt, J.K. and Ceulemans, E. (2022). Mixture multigroup factor analysis for unraveling factor loading noninvariance across many groups. Psychological Methods, 27(3):281–306.
Diaconis, P. and Efron, B. (1983). Computer-intensive methods in statistics. Scientific American, 248(5):116–131.
Fischer, A., Gaunt, R.E. and Sarantsev, A. (2023). The variance-gamma distribution: A review. arXiv preprint arXiv:230305615.
Gaarenstroom, P.D., Perone, S.P. and Moyers, J.L. (1977). Application of pattern recognition and factor analysis for characterization of atmospheric particulate composition in southwest desert atmosphere. Environmental Science & Technology, 11(8):795–800.
Ghahramani, Z. and Hinton, G.E. (1997). The EM Algorithm for Mixtures of Factor Analyzers. Vol. 60, Technical Report CRG-TR-96-1. University of Toronto.
Ghitany, M.E., Atieh, B. and Nadarajah, S. (2008). Lindley distribution and its application. Mathematics and Computers in Simulation, 78(4):493–506.
Göncü, A. and Yang, H. (2016). Variance-gamma and normal-inverse Gaussian models: Goodness-of-fit to Chinese high-frequency index returns. The North American Journal of Economics and Finance, 36:279–292.
Good, I.J. (1953). The population frequencies of species and the estimation of population parameters. Biometrika, 40(3/4):237–264.
Hartigan, J.A. and Wong, M.A. (1979). Algorithm AS 136: A K-means clustering algorithm. Journal of the Royal Statistical Society, Series C (Applied Statistics), 28(1):100–108.
Hashemi, F., Naderi, M., Jamalizadeh, A. and Lin, T. (2020). A skew factor analysis model based on the normal mean-variance mixture of Birnbaum-Saunders distribution. Journal of Applied Statistics, 47(16):3007–3029.
Hubert, L. and Arabie, P. (1985). Comparing partitions. Journal of Classification, 2(1):193–218.
Joreskog, K.G., Sorbom, D. and Magidson, J. (1979). Advances in Factor Analysis and Structural Equation Models. Abt books.
Keribin, C. (2000). Consistent estimation of the order of mixture models. Sankhyā: The Indian Journal of Statistics, Series A, 62:49–66.
Lawley, D.N. and Maxwell, A.E. (1962). Factor analysis as a statistical method. Journal of the Royal Statistical Society, Series D (The Statistician), 12(3):209–229.
Lee, S.X. and McLachlan, G.J. (2018). On formulations of skew factor models: Skew errors versus skew factors. arXiv preprint arXiv:1810.04842.
Lee, S.X., Lin, T.I. and McLachlan, G.J. (2018). Mixtures of factor analyzers with fundamental skew symmetric distributions. arXiv preprint arXiv:1802.02467.
Lee, S.X., Lin, T.I. and McLachlan, G.J. (2021). Mixtures of factor analyzers with scale mixtures of fundamental skew normal distributions. Advances in Data Analysis and Classification, 15(2):481–512.
Lin, T.I., McLachlan, G.J. and Lee, S.X. (2016). Extending mixtures of factor models using the restricted multivariate skew-normal distribution. Journal of Multivariate Analysis, 143:398–413.
Lin, T.I., Wang, W.L., McLachlan, G.J. and Lee, S.X. (2018). Robust mixtures of factor analysis models using the restricted multivariate skew-t distribution. Statistical Modelling, 18(1):50–72.
Lin, T.I., Wu, P.H., McLachlan, G.J. and Lee, S.X. (2015). A robust factor analysis model using the restricted skew-t distribution. Test, 24(3):510–531.
Little, R.J. and Rubin, D.B. (2019). Statistical Analysis with Missing Data. John Wiley & Sons.
Liu, C. and Rubin, D.B. (1994). The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence. Biometrika, 81(4):633–648.
Lopes, H.F. and West, M. (2004). Bayesian model assessment in factor analysis. Statistica Sinica, 14(1):41–67.
Maleki, M. and Wraith, D. (2019). Mixtures of multivariate restricted skew-normal factor analyzer models in a Bayesian framework. Computational Statistics, 34(3):1039–1053.
McLachlan, G.J. and Krishnan, T. (2007). The EM algorithm and extensions, 2nd edition. John Wiley & Sons.
McLachlan, G.J., Bean, R.W. and Jones, L.B.T. (2007). Extension of the mixture of factor analyzers model to incorporate the multivariate t-distribution. Computational Statistics & Data Analysis, 51(11):5327–5338.
McLachlan, G.J., Peel, D. and Bean, R.W. (2003). Modelling high-dimensional data by mixtures of factor analyzers. Computational Statistics & Data Analysis, 41(3-4):379–388.
McNicholas, S.M., McNicholas, P.D. and Browne, R.P. (2017). A mixture of variance-gamma factor analyzers. Big and Complex Data Analysis: Methodologies and Applications, 369–385.
Meng, X. and Rubin, D.B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika, 80(2):267–278.
Murray, P.M., Browne, R.P. and McNicholas, P.D. (2013). Mixtures of unrestricted skew-t factor analyzers. arXiv preprint arXiv:1310.6224.
Murray, P.M., Browne, R.P. and McNicholas, P.D. (2014). Mixtures of skew-t factor analyzers. Computational Statistics & Data Analysis, 77:326–335.
Murray, P.M., McNicholas, P.D. and Browne, R.P. (2014). A mixture of common skew-t factor analysers. Stat, 3(1):68–82.
Naderi, M., Arabpour, A. and Jamalizadeh, A. (2018). Multivariate normal mean-variance mixture distribution based on Lindley distribution. Communications in Statistics-Simulation and Computation, 47(4):1179–1192.
Rand, W.M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336):846–850.
Rubin, D.B. (1976). Inference and missing data. Biometrika, 63(3):581–592.
Schafer, J.L. (1997). Analysis of Incomplete Multivariate Data. CRC press.
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2):461–464.
Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 3/4(1987):441–471.
Tortora, C., McNicholas, P.D. and Browne, R.P. (2016). A mixture of generalized hyperbolic factor analyzers. Advances in Data Analysis and Classification, 10(4):423–440.
Wall, M.M., Guo, J. and Amemiya, Y. (2012). Mixture factor analysis for approximating a nonnormally distributed continuous latent factor with continuous and dichotomous observed variables. Multivariate Behavioral Research, 47(2):276–313.
Wang, W.L. (2013). Mixtures of common factor analyzers for high-dimensional data with missing information. Journal of Multivariate Analysis, 117:120–133.
Wang, W.L. (2015). Mixtures of common t-factor analyzers for modeling high-dimensional data with missing values. Computational Statistics & Data Analysis, 83:223–235.
Wang, W.L., Liu, M. and Lin, T.I. (2017). Robust skew-t factor analysis models for handling missing data. Statistical Methods & Applications, 26:649–672.
Wei, Y., Tang, Y. and McNicholas, P.D. (2018). Flexible high-dimensional unsupervised learning with missing data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(3):610–621.
Journal of Statistical Modelling: Theory and Applications
Volume 4, Issue 2
July 2023
Pages 35-56

Files

History

  • Receive Date: 02 February 2024
  • Revise Date: 17 May 2024
  • Accept Date: 30 May 2024

Share

How to cite

Statistics