Robust mixture of experts modeling using symmetric α-stable distributions

Document Type : Original Scientific Paper

Author

Department of Statistics‎, ‎Faculty of Science‎, ‎University of Kurdistan‎, ‎Sanandaj‎, ‎Iran

Abstract

‎The mixture of experts framework is widely utilized in statistics and machine learning to address data heterogeneity in tasks such as regression, classification, and clustering. In clustering continuous data, the mixture of experts typically employs experts that follow a Gaussian distribution. However, outliers can adversely affect clustering outcomes. To address this issue, various methods have been proposed in the literature. In this paper, we introduce a novel approach that models the experts using the symmetric α-stable distribution. This flexible distribution effectively accommodates different types of outliers (especially extreme outliers) and skewness, while also encompassing Gaussian experts as a special case when α =2. The maximum likelihood estimates of the model parameters (excluding α) are obtained using an expectation-maximization approach, while α is estimated using Monte Carlo integration and interpolation. The effectiveness of this approach is demonstrated through analyses of both real and simulated data.‎

Keywords

Main Subjects

References

Bai, X., Yao, W. and Boyer, J.E. (2012). Robust fitting of mixture regression models. Computational Statistics & Data Analysis, 56(7):2347–2359.
Celeux, G. and Diebolt, J. (1985). The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Computational Statistics, 2(1):73–82.
Chamroukhi, F. (2016). Robust mixture of experts modeling using the t distribution. Neural Networks, 79:20–36.
Chamroukhi, F., Lecocq, F. and Bartcus, M. (2019). Meteorits: Mixtures-of-experts modeling for complex and non-normal. R package version 0.1.1. https://github.com/fchamroukhi/MEteorits.
Cohen, E.A. (1984). Some effects of inharmonic partials on interval perception. Music Perception, 1(3):323–349.
Farcomeni, A. and Punzo, A. (2020). Robust model-based clustering with mild and gross outliers. Test, 29(4):989–1007.
Goldfeld, S.M. and Quandt, R.E. (1973). A Markov model for switching regression. Journal of Econometrics, 3(1):3–15.
Hubert, L. and Arabie, P. (1985). Comparing partitions. Journal of Classification, 2(1):193–218.
Jacobs, R.A., Jordan, M.I., Nowlan, S.J. and Hinton, G.E. (1991). Adaptive mixtures of local experts. Neural Computation, 3(1):79–87.
Jain, A.K. and Dubes, R.C. (1988). Algorithms for Clustering Data. New York: Prentice and Hall.
Jiang, W. and Tanner, M.A. (1999). On the identifiability of mixtures-of-experts. Neural Networks, 12(9):197–220.
Jordan, M.I. and Jacobs, R.A. (1994). Hierarchical mixtures of experts and the EM algorithm. Computational Statistics, 6(2):181–214.
Kong, A., McCullagh, P., Meng, X.L., Nicolae, D. and Tan, Z. (2009). A theory of statistical models for Monte Carlo integration. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65(3):585–604.
Kring, S., Rachev, S.T., Höchstötter, M. and Fabozzi, F.J. (2009). Estimation of α-stable sub-Gaussian distributions for asset returns. Risk Assessment, 12(2):111–152.
Lévy, P. (1925). Calcul des probabilites. Paris: Gauthier-Villars.
Mandelbrot, B. (1961). Stable paretian random functions and the multiplicative variation of income. Econometrica, 29(4):517–543.
Mandelbrot, B. (1963b). New methods in statistical economics. The Journal of Political Economy, 71(5):421–440.
McLachlan, G.J. and Peel, D. (2000). Finite Mixture Models. New York: John Wiley & Sons.
Nguyen, H.D. and McLachlan, G.J. (2016). Laplace mixture of linear experts. Computational Statistics & Data Analysis, 93:177–191.
Nolan, J.P. (2009). Stable Distributions: Models for Heavy Tailed Data. Boston: Birkhauser.
Ritter, G. (2014). Robust Cluster Analysis and Variable Selection. Vol. 137, Chapman & Hall/CRC Monographs on Statistics & Applied Probability.
Roche, A. (2011). EM algorithm and variants: An informal tutorial. arXiv preprint arXiv:1105.1476.
Salas-Gonzalez, D., Kuruoglu, E.E. and Ruiz, D.P. (2009). Finite mixture of α-stable distributions. Digital Signal Processing, 19(2):250–264.
Samorodnitsky, G. and Taqqu, M.S. (1994). Stable Non-Gaussian Random Processes. New York: Chapman and Hall.
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2):461–464.
Song, W., Yao, W. and Xing, Y. (2014). Robust mixture regression model fitting by Laplace distribution. Computational Statistics & Data Analysis, 71(10):128–137.
Yao, W., Wei, Y. and Yu, C. (2014). Robust mixture regression using the t-distribution. Computational Statistics & Data Analysis, 71(1):116–127.
Yuksel, S.E., Wilson, J.N. and Gader, P.D. (2012). Twenty years of mixture of experts. IEEE Transactions on Neural Networks and Learning Systems, 28(8):1177–1193.
Zarei, S. (2024). Robust mixture of regression models using the symmetric α-stable distribution. Communications in Statistics-Simulation and Computation, 54(10):3879–3897.
Zarei, S. and Mohammdpour, A. (2020). Pseudo-stochastic EM for sub-Gaussian α-stable mixture models. Digital Signal Processing, 99:102671.
Zarei, S., Mohammadpour, A., Ingrassia, S. and Punzo, A. (2019). On the use of the sub-Gaussian α-stable distribution in the cluster-weighted model. Iranian Journal of Science and Technology, Transactions A: Science, 43(3):1059–1069.
Journal of Statistical Modelling: Theory and Applications
Volume 6, Issue 1
January 2025
Pages 133-150

Files

History

  • Receive Date: 29 April 2025
  • Revise Date: 31 August 2025
  • Accept Date: 23 December 2025

Share

How to cite

Statistics