Selected work

Representative Papers

B3O: Scalable Boltzmann Batch Bayesian Optimization

Jun 29, 2026

This work addresses the high computational cost and limited batch diversity in large-scale parallel Bayesian optimization by reframing batch generation as a pure sampling problem—specifically, direct sampling from the Boltzmann distribution induced by the acquisition function. This approach circumvents the computational bottlenecks inherent in conventional large-batch optimization schemes. Theoretically, the sampled points introduce only negligible additional regret while ensuring high diversity. Empirical evaluations demonstrate that the proposed method outperforms existing techniques on standard synthetic benchmarks and exhibits superior performance and robustness in complex real-world tasks, including multi-objective electrode design and mixed-variable racecar configuration.

0 citationsRead paper

Estimating Aleatoric Uncertainty in the Causal Treatment Effect

Feb 09, 2026

This study addresses a critical limitation in existing causal inference methods, which predominantly focus on average treatment effects while neglecting the stochastic variability in individual responses. To capture this uncertainty, the paper introduces the variance of treatment effects (VTE) and the conditional variance of treatment effects (CVTE) as central measures of causal response heterogeneity. Under relatively weak assumptions allowing for unobserved confounding, the authors establish the identifiability of these variance measures and propose a consistent nonparametric kernel-based estimation framework. Theoretical analysis demonstrates the convergence properties of the proposed estimators, and experiments on synthetic and semi-simulated data show that the method achieves estimation accuracy that either matches or surpasses current baselines, thereby moving beyond the conventional mean-centric paradigm in causal inference.

0 citationsRead paper

LLM Flow Processes for Text-Conditioned Regression

Jan 05, 2026arXiv.org

This work proposes a general framework that addresses the limitations of existing meta-learning regression methods, which struggle to effectively integrate expert priors with textual metadata, and the constrained regression performance of large language models (LLMs) despite their rich semantic capabilities. The approach uniquely combines the text-conditional probability outputs of LLMs with neural diffusion or flow-matching processes through a product-of-experts mechanism, enabling joint sampling between a binning-based probabilistic density “expert” model and a diffusion generative model. By doing so, the method preserves the semantic understanding of LLMs while significantly enhancing regression accuracy. Empirical evaluations across multiple benchmark tasks demonstrate consistent superiority over both standalone LLMs and neural process–based approaches, validating the effective contribution of text-conditioned knowledge to regression performance.

0 citationsRead paper

Omnipresent Yet Overlooked: Heat Kernels in Combinatorial Bayesian Optimization

Oct 30, 2025

Bayesian optimization (BO) for combinatorial domains suffers from a lack of unified theoretical foundations for kernel design. To address this, we propose a principled framework based on the heat kernel, establishing for the first time a systematic connection between combinatorial kernels and graph heat kernels. We prove that the resulting kernel is insensitive to optimal solution location and exhibits strong structural invariance. By integrating graph-theoretic principles with diffusion process modeling, we derive compact closed-form heat kernels applicable to diverse combinatorial structures—including sequences, trees, and graphs—and seamlessly embed them into standard BO pipelines. Theoretical analysis reveals that several existing combinatorial kernels are special cases of our heat kernel formulation. Experiments on benchmark tasks—including materials discovery and neural architecture search—demonstrate state-of-the-art performance, significantly outperforming both complex and computationally intensive baselines.

0 citationsRead paper

Closed-Form Last Layer Optimization

Oct 06, 2025

Standard neural network training suffers from inefficient optimization of the final-layer linear weights. Method: Under squared loss, this work explicitly leverages the closed-form optimal solution of the last-layer weights with respect to backbone parameters, treating them as an analytic function of the backbone and performing gradient updates solely on the backbone. This approach strictly embeds the closed-form solution into training—equivalent to alternating optimization between backbone and last layer—and establishes convergence within the Neural Tangent Kernel (NTK) framework. Contribution/Results: By integrating the analytic solution with stochastic gradient descent (SGD), the method dynamically balances per-batch loss minimization against historical information. It significantly outperforms standard SGD on diverse tasks—including Fourier neural operators and instrumental variable regression—while remaining applicable to both regression and classification.

0 citationsRead paper

Recent publications

Latest Papers

B3O: Scalable Boltzmann Batch Bayesian Optimization

Jun 29, 2026

This work addresses the high computational cost and limited batch diversity in large-scale parallel Bayesian optimization by reframing batch generation as a pure sampling problem—specifically, direct sampling from the Boltzmann distribution induced by the acquisition function. This approach circumvents the computational bottlenecks inherent in conventional large-batch optimization schemes. Theoretically, the sampled points introduce only negligible additional regret while ensuring high diversity. Empirical evaluations demonstrate that the proposed method outperforms existing techniques on standard synthetic benchmarks and exhibits superior performance and robustness in complex real-world tasks, including multi-objective electrode design and mixed-variable racecar configuration.

0 citationsRead paper

Estimating Aleatoric Uncertainty in the Causal Treatment Effect

Feb 09, 2026

This study addresses a critical limitation in existing causal inference methods, which predominantly focus on average treatment effects while neglecting the stochastic variability in individual responses. To capture this uncertainty, the paper introduces the variance of treatment effects (VTE) and the conditional variance of treatment effects (CVTE) as central measures of causal response heterogeneity. Under relatively weak assumptions allowing for unobserved confounding, the authors establish the identifiability of these variance measures and propose a consistent nonparametric kernel-based estimation framework. Theoretical analysis demonstrates the convergence properties of the proposed estimators, and experiments on synthetic and semi-simulated data show that the method achieves estimation accuracy that either matches or surpasses current baselines, thereby moving beyond the conventional mean-centric paradigm in causal inference.

0 citationsRead paper

LLM Flow Processes for Text-Conditioned Regression

Jan 05, 2026arXiv.org

This work proposes a general framework that addresses the limitations of existing meta-learning regression methods, which struggle to effectively integrate expert priors with textual metadata, and the constrained regression performance of large language models (LLMs) despite their rich semantic capabilities. The approach uniquely combines the text-conditional probability outputs of LLMs with neural diffusion or flow-matching processes through a product-of-experts mechanism, enabling joint sampling between a binning-based probabilistic density “expert” model and a diffusion generative model. By doing so, the method preserves the semantic understanding of LLMs while significantly enhancing regression accuracy. Empirical evaluations across multiple benchmark tasks demonstrate consistent superiority over both standalone LLMs and neural process–based approaches, validating the effective contribution of text-conditioned knowledge to regression performance.

0 citationsRead paper

Omnipresent Yet Overlooked: Heat Kernels in Combinatorial Bayesian Optimization

Oct 30, 2025

Bayesian optimization (BO) for combinatorial domains suffers from a lack of unified theoretical foundations for kernel design. To address this, we propose a principled framework based on the heat kernel, establishing for the first time a systematic connection between combinatorial kernels and graph heat kernels. We prove that the resulting kernel is insensitive to optimal solution location and exhibits strong structural invariance. By integrating graph-theoretic principles with diffusion process modeling, we derive compact closed-form heat kernels applicable to diverse combinatorial structures—including sequences, trees, and graphs—and seamlessly embed them into standard BO pipelines. Theoretical analysis reveals that several existing combinatorial kernels are special cases of our heat kernel formulation. Experiments on benchmark tasks—including materials discovery and neural architecture search—demonstrate state-of-the-art performance, significantly outperforming both complex and computationally intensive baselines.

0 citationsRead paper

Closed-Form Last Layer Optimization

Oct 06, 2025

Standard neural network training suffers from inefficient optimization of the final-layer linear weights. Method: Under squared loss, this work explicitly leverages the closed-form optimal solution of the last-layer weights with respect to backbone parameters, treating them as an analytic function of the backbone and performing gradient updates solely on the backbone. This approach strictly embeds the closed-form solution into training—equivalent to alternating optimization between backbone and last layer—and establishes convergence within the Neural Tangent Kernel (NTK) framework. Contribution/Results: By integrating the analytic solution with stochastic gradient descent (SGD), the method dynamically balances per-batch loss minimization against historical information. It significantly outperforms standard SGD on diverse tasks—including Fourier neural operators and instrumental variable regression—while remaining applicable to both regression and classification.

0 citationsRead paper