Bridging the Prediction Error Method and Subspace Identification: A Weighted Null Space Fitting Method

2025-10-02

AI Summary

To address the low statistical efficiency of subspace identification methods (SIMs) and the non-convex optimization and initialization sensitivity inherent in prediction-error methods (PEMs), this paper proposes the Weighted Null-Space Fitting (WNSF) method. WNSF begins with a weighted least-squares estimate of a high-order ARX model and constructs a state-space realization via null-space projection and multi-stage dimensionality reduction. Under canonical parametrization, it achieves asymptotic efficiency—i.e., attains the Cramér–Rao lower bound of maximum likelihood estimation (MLE)—for the first time among subspace-type methods. Theoretical analysis establishes consistency and asymptotic efficiency; numerical experiments demonstrate that WNSF significantly outperforms conventional SIMs in accuracy, closely approaches MLE performance, and avoids PEM’s optimization difficulties. This work resolves the long-standing statistical efficiency bottleneck of subspace methods, establishing a new paradigm for linear system identification that jointly ensures robustness, tractability, and optimal statistical properties.

Technology Category

• Intelligent Robots: State Estimation
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Matrix & Tensor Methods

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Subspace identification methods (SIMs) have proven to be very useful and numerically robust for building state-space models. While most SIMs are consistent, few if any can achieve the efficiency of the maximum likelihood estimate (MLE). Conversely, the prediction error method (PEM) with a quadratic criteria is equivalent to MLE, but it comes with non-convex optimization problems and requires good initialization points. This contribution proposes a weighted null space fitting (WNSF) approach for estimating state-space models, combining some key advantages of the two aforementioned mainstream approaches. It starts with a least-squares estimate of a high-order ARX model, and then a multi-step least-squares procedure reduces the model to a state-space model on canoncial form. It is demonstrated through statistical analysis that when a canonical parameterization is admissible, the proposed method is consistent and asymptotically efficient, thereby making progress on the long-standing open problem about the existence of an asymptotically efficient SIM. Numerical and practical examples are provided to illustrate that the proposed method performs favorable in comparison with SIMs.

Problem

• Combining subspace identification and prediction error methods for efficiency
• Achieving asymptotic efficiency in state-space model identification
• Avoiding non-convex optimization problems in maximum likelihood estimation

Innovation

• Combines prediction error and subspace identification methods
• Uses weighted null space fitting for state-space estimation
• Starts with high-order ARX model then reduces complexity