Bounded-Rationality, Hedging, and Generalization

2026-05-14

AI Summary

This work investigates how to quantify a learner’s dependence on training samples from their black-box response behavior and establish verifiable generalization guarantees. Modeling the learning process as a bounded-rational decision problem, it characterizes the mapping from samples to outputs via an induced channel and leverages response laws to reveal the trade-off between training loss and sample dependence. The study innovatively interprets generalization as a testable hedging property of the response law and constructs an information-geometric framework based on f-divergence regularization, unifying classical cases such as KL divergence and mutual information. By analyzing local perturbations in loss and channel, the approach recovers the underlying hedging mechanism from observed behavior, yielding certified upper and lower bounds on generalization error, and provides practical generalization guarantees under conditions that account for sample-induced distortion.

Abstract

A learner does not only fit data; it also determines how strongly the training sample may shape its output and how much distortion it can hedge. We study this relation as a bounded-rational decision problem whose primitive object is the induced channel from samples to outputs. The learner's response law determines which changes in this channel are cheap or costly, and therefore induces both a lower tradeoff curve between training loss and sample dependence and a matched upper certificate curve. When the response law is represented by an $f$-divergence regularizer, these curves live in the regularizer's native information geometry, with KL as the special case corresponding to Shannon mutual information. We show how the hedge and the two curves can be recovered from black-box behavior by observing responses to scaled losses and local loss perturbations. In learning, population loss is empirical loss plus the distortion induced by the particular training sample. The recovered hedge gives a practical certificate when it covers that distortion. Thus generalization is treated as a testable hedging property of the learner's own response law.

Problem

• Bounded Rationality
• Hedging
• Generalization
• f-divergence
• Information Geometry

Innovation

• Bounded Rationality
• Hedging
• f-divergence
• Information Geometry
• Generalization Certificate