🤖 AI Summary
High-order Markov chains suffer from overfitting and identifiability issues due to exponential growth of parameters with order. Method: We propose a Sparse Markov Model (SMM) estimation framework integrating convex clustering and mixed L₁/L₂ regularization. It is the first to incorporate convex clustering into SMM structure learning, automatically grouping historical state sequences and sharing transition probability vectors within clusters. The method jointly optimizes maximum likelihood estimation with mixed regularization and employs the Bayesian Information Criterion (BIC) to adaptively tune regularization strength. Theoretically, it achieves model selection consistency under large-sample asymptotics. Results: Experiments demonstrate superior performance over baselines—including Variable-Length Markov Chains (VLMC)—on multiple synthetic datasets, and yield improved classification accuracy on viral genome sequence analysis.
📝 Abstract
The major problem of fitting a higher order Markov model is the exponentially growing number of parameters. The most popular approach is to use a Variable Length Markov Chain (VLMC), which determines relevant contexts (recent pasts) of variable orders and form a context tree. A more general approach is called Sparse Markov Model (SMM), where all possible histories of order $m$ form a partition so that the transition probability vectors are identical for the histories belonging to a particular group. We develop an elegant method of fitting SMM using convex clustering, which involves regularization. The regularization parameter is selected using BIC criterion. Theoretical results demonstrate the model selection consistency of our method for large sample size. Extensive simulation studies under different set-up have been presented to measure the performance of our method. We apply this method to classify genome sequences, obtained from individuals affected by different viruses.
