Information Geometry of Message Passing
This study addresses the accuracy limitations of variational inference in non-conjugate and persistent uncertainty settings by proposing the Natural Gradient Message Passing (NGMP) algorithm. Grounded in information geometry and Forney-style factor graphs, NGMP reformulates stationarity conditions into edge-local forms and employs natural gradient projections to preserve exact messages representable within the receiving family, thereby effectively mitigating precision loss caused by averaging in traditional methods. Empirical evaluations across Poisson smoothing, heteroscedastic regression, and ETTH forecasting tasks demonstrate that NGMP significantly enhances both uncertainty calibration and inference performance in non-conjugate models. These results establish NGMP as a robust approach for improving variational inference accuracy where standard mean-field approximations typically fail due to structural mismatches or sustained uncertainty propagation.