Here Be Dragons: Bimodal posteriors arise from numerical integration error in longitudinal models

2025-02-17

AI Summary

This work reveals that numerical integration errors in longitudinal differential equation models can induce spurious bimodality in Bayesian posterior distributions: one mode near the true parameter value, the other far from it yet prediction-equivalent, thereby undermining parameter identifiability. To address this, we first provide a rigorous theoretical analysis showing how Runge–Kutta–type integrators systematically induce bimodality in affine first-order ODEs. We then propose a testable simulation-based diagnostic procedure and an MCMC correction strategy to mitigate the artifact. Theoretical analysis and extensive simulations jointly validate the diagnostic’s efficacy; moreover, the corrected MCMC sampler successfully eliminates the spurious mode. Our contributions thus provide both a critical error-detection tool and a robust inference framework for Bayesian inverse problems reliant on numerical ODE solvers.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Longitudinal models with dynamics governed by differential equations may require numerical integration alongside parameter estimation. We have identified a situation where the numerical integration introduces error in such a way that it becomes a novel source of non-uniqueness in estimation. We obtain two very different sets of parameters, one of which is a good estimate of the true values and the other a very poor one. The two estimates have forward numerical projections statistically indistinguishable from each other because of numerical error. In such cases, the posterior distribution for parameters is bimodal, with a dominant mode closer to the true parameter value, and a second cluster around the errant value. We demonstrate that multi-modality exists both theoretically and empirically for an affine first order differential equation, that a simulation workflow can test for evidence of the issue more generally, and that Markov Chain Monte Carlo sampling with a suitable solution can avoid bimodality. The issue of multi-modal posteriors arising from numerical error has consequences for Bayesian inverse methods that rely on numerical integration more broadly.

Problem

• Numerical integration errors cause non-uniqueness in parameter estimation
• Bimodal posteriors arise from indistinguishable forward projections due to errors
• Multi-modality impacts Bayesian inverse methods relying on numerical integration

Innovation

• Numerical integration causes bimodal posterior distributions
• Simulation workflow detects numerical error issues
• MCMC sampling avoids bimodality with suitable solutions