Function-on-function Differential Regression

2025-06-03

AI Summary

Physical systems often exhibit inherent differential relationships, yet conventional function-to-function regression frameworks rely predominantly on integral-type operators, limiting their ability to directly model differential dynamics. Method: This paper proposes a novel operator-valued function-to-function regression framework explicitly designed for differential operators—departing from traditional integral-based paradigms. It introduces the first “action-aware” differential operator identification paradigm and establishes a rigorous theoretical foundation within the Operator-Valued Reproducing Kernel Hilbert Space (ORKHS), encompassing regularized estimation and goodness-of-fit testing for differential operators. Contribution/Results: The estimator is proven minimax optimal; the proposed test is shown to be uniformly consistent. Empirically validated on the thermodynamic energy equation, the method achieves significantly improved predictive accuracy and enhanced physical interpretability. It provides a principled, interpretable, and statistically verifiable functional modeling paradigm for differential-equation-driven scientific machine learning.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Function-on-function regression has been a topic of substantial interest due to its broad applicability, where the relation between functional predictor and response is concerned. In this article, we propose a new framework for modeling the regression mapping that extends beyond integral type, motivated by the prevalence of physical phenomena governed by differential relations, which is referred to as function-on-function differential regression. However, a key challenge lies in representing the differential regression operator, unlike functions that can be expressed by expansions. As a main contribution, we introduce a new notion of model identification involving differential operators, defined through their action on functions. Based on this action-aware identification, we are able to develop a regularization method for estimation using operator reproducing kernel Hilbert spaces. Then a goodness-of-fit test is constructed, which facilitates model checking for differential regression relations. We establish a Bahadur representation for the regression estimator with various theoretical implications, such as the minimax optimality of the proposed estimator, and the validity and consistency of the proposed test. To illustrate the effectiveness of our method, we conduct simulation studies and an application to a real data example on the thermodynamic energy equation.

Problem

• Modeling regression mapping beyond integral type
• Representing differential regression operator effectively
• Developing regularization method for operator estimation

Innovation

• Function-on-function differential regression framework
• Operator reproducing kernel Hilbert spaces
• Goodness-of-fit test for differential relations