Selected work

Representative Papers

Applying Embedding-Based Retrieval to Airbnb Search

Jan 11, 2026arXiv.org

This work proposes an embedding-based retrieval (EBR) system to address the challenge of efficient search in Airbnb’s dynamic marketplace, characterized by vast and evolving inventory, heterogeneous user preferences, and multi-product scenarios such as flexible-date searches. The approach leverages deep learning models to generate semantic embeddings for both queries and listings, enabling low-latency candidate retrieval via approximate nearest neighbor search. Innovatively, the system captures key characteristics of a two-sided marketplace—including dynamic inventory availability, long conversion funnels, and diverse retrieval contexts—and integrates seamlessly into a multi-stage ranking architecture. Upon deployment, the EBR system significantly improved booking conversion rates and has been successfully applied across core business scenarios, including standard search, flexible-date search, and personalized marketing emails.

1 citationsRead paper

Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses

Sep 04, 2026

LLM tool agents can be improved without retraining by modifying the runtime harness around a fixed model: prompts, tool interfaces, middleware, state handling, and recovery logic. We study this setting as resource-bounded harness selection for fixed-model multi-turn tool agents, with the search surface scoped to prompts and tool-boundary middleware: edits are guarded intercepts at the tool boundary, not arbitrary rewriting of agent execution logic. Our optimizer-agnostic protocol reports mean held-out lift, worst-condition lift, repeatability, logged cost diagnostics, and RelLift95(B), a conservative estimate of the held-out gain of the harness selected under budget B. We instantiate the protocol with prompt-only and prompt-plus-middleware optimizers, including PRISM, which clusters failures and routes repairs to prompt, tool-boundary middleware, or joint edit surfaces within a Pareto search. On BFCL multi-round, tau2-Retail, and tau2-Telecom, PRISM obtains mean held-out lifts of 14.2, 14.9, and 10.1 percentage points and positive empirical RelLift95 on all three benchmarks, and a component ablation attributes the margin chiefly to failure-surface routing and the edit-pattern constraint. Across optimizers, the results show that some search procedures can occasionally find large gains but still choose brittle updates, so the reliability of the chosen harness should be reported alongside average held-out lift.

0 citationsRead paper

From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale

Sep 04, 2026

Conversational assistants can blend retrieval, action selection, escalation, and wording in a single model path, or separate those roles. We report a production migration of a customer-support assistant at a large accommodation marketplace (millions of conversations per month, 11 languages, 10-second P90). Dynamic Response (DR) replaces a single Qwen3-235B-A22B blended responder with a bounded ReAct orchestrator over typed tools plus a smaller generator that writes from a backend-validated context contract. Because the migration also changed prompts, alignment, and serving, we attribute each effect to its cause and claim as architecture effects only those measured on identical replayed turns: typed entity selection moves the reservation selector to a precision-first operating point (precision 8.3% to 89.1%, recall 75.2% to 67.3%), and typed action IDs with a membership check remove observed structured-action hallucination (2.14% to 0.0%). A low-ramp A/B test reproduces the replay escalation reductions: hard-escalation responses fall from 5.60% to 3.08% and soft-escalation responses from 9.56% to 2.49%, while production handoff volume holds roughly steady; self-solve is directional (+5.1 points, 95% CI [-2, +12]). Serving optimizations cut orchestrator P90 latency from 3.87s to 2.24s on a GPU footprint reduced by roughly one-third, and self-hosting reduces estimated annual model-serving cost by more than an order of magnitude.

0 citationsRead paper

Evaluating for the long term: Learnings from industry

Aug 08, 2026

Short-term online experiments often fail to accurately predict long-term business outcomes, potentially leading to decisions misaligned with strategic objectives. Drawing on insights from industry expert workshops, this work proposes a design principle for surrogate metrics centered on decision utility rather than solely on unbiasedness, advocating that interpretable, experiment-driven simple surrogates outperform complex black-box models. Through expert consensus synthesis, surrogate metric analysis, and comparative evaluation of experimental versus observational data, the study systematically outlines a methodology for constructing effective surrogates and uncovers a critical relationship between the stability of long-term effects and surrogate validity. While underscoring the irreplaceable value of high-quality long-term experimentation, the research also delineates core challenges and practical guidelines for surrogate learning in real-world settings.

0 citationsRead paper

Proximity Features: Privacy-Compliant Cold-Start Personalization at Airbnb

Jul 13, 2026

This study addresses the challenge of personalizing recommendations for new or unauthenticated users in Airbnb’s cold-start scenarios, particularly under stringent privacy regulations that prohibit the use of persistent individual identifiers. To overcome this limitation, the authors propose a geolocation-based adaptive clustering method that dynamically aggregates approximately 1,000 proximate users into cohorts, thereby constructing group-level feature signals without relying on persistent user identities. This approach introduces a novel privacy-compliant mechanism driven by geographic proximity to enable effective personalization while preserving user anonymity. Online A/B experiments demonstrate that the proposed method significantly improves booking conversion rates, with especially pronounced gains for users exhibiting no prior activity or outdated behavioral data. The solution has been successfully deployed in production systems, including marketing landing pages and destination recommendation modules.

0 citationsRead paper

Recent publications

Latest Papers

Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses

Sep 04, 2026

LLM tool agents can be improved without retraining by modifying the runtime harness around a fixed model: prompts, tool interfaces, middleware, state handling, and recovery logic. We study this setting as resource-bounded harness selection for fixed-model multi-turn tool agents, with the search surface scoped to prompts and tool-boundary middleware: edits are guarded intercepts at the tool boundary, not arbitrary rewriting of agent execution logic. Our optimizer-agnostic protocol reports mean held-out lift, worst-condition lift, repeatability, logged cost diagnostics, and RelLift95(B), a conservative estimate of the held-out gain of the harness selected under budget B. We instantiate the protocol with prompt-only and prompt-plus-middleware optimizers, including PRISM, which clusters failures and routes repairs to prompt, tool-boundary middleware, or joint edit surfaces within a Pareto search. On BFCL multi-round, tau2-Retail, and tau2-Telecom, PRISM obtains mean held-out lifts of 14.2, 14.9, and 10.1 percentage points and positive empirical RelLift95 on all three benchmarks, and a component ablation attributes the margin chiefly to failure-surface routing and the edit-pattern constraint. Across optimizers, the results show that some search procedures can occasionally find large gains but still choose brittle updates, so the reliability of the chosen harness should be reported alongside average held-out lift.

0 citationsRead paper

From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale

Sep 04, 2026

Conversational assistants can blend retrieval, action selection, escalation, and wording in a single model path, or separate those roles. We report a production migration of a customer-support assistant at a large accommodation marketplace (millions of conversations per month, 11 languages, 10-second P90). Dynamic Response (DR) replaces a single Qwen3-235B-A22B blended responder with a bounded ReAct orchestrator over typed tools plus a smaller generator that writes from a backend-validated context contract. Because the migration also changed prompts, alignment, and serving, we attribute each effect to its cause and claim as architecture effects only those measured on identical replayed turns: typed entity selection moves the reservation selector to a precision-first operating point (precision 8.3% to 89.1%, recall 75.2% to 67.3%), and typed action IDs with a membership check remove observed structured-action hallucination (2.14% to 0.0%). A low-ramp A/B test reproduces the replay escalation reductions: hard-escalation responses fall from 5.60% to 3.08% and soft-escalation responses from 9.56% to 2.49%, while production handoff volume holds roughly steady; self-solve is directional (+5.1 points, 95% CI [-2, +12]). Serving optimizations cut orchestrator P90 latency from 3.87s to 2.24s on a GPU footprint reduced by roughly one-third, and self-hosting reduces estimated annual model-serving cost by more than an order of magnitude.

0 citationsRead paper

Evaluating for the long term: Learnings from industry

Aug 08, 2026

Short-term online experiments often fail to accurately predict long-term business outcomes, potentially leading to decisions misaligned with strategic objectives. Drawing on insights from industry expert workshops, this work proposes a design principle for surrogate metrics centered on decision utility rather than solely on unbiasedness, advocating that interpretable, experiment-driven simple surrogates outperform complex black-box models. Through expert consensus synthesis, surrogate metric analysis, and comparative evaluation of experimental versus observational data, the study systematically outlines a methodology for constructing effective surrogates and uncovers a critical relationship between the stability of long-term effects and surrogate validity. While underscoring the irreplaceable value of high-quality long-term experimentation, the research also delineates core challenges and practical guidelines for surrogate learning in real-world settings.

0 citationsRead paper

Proximity Features: Privacy-Compliant Cold-Start Personalization at Airbnb

Jul 13, 2026

This study addresses the challenge of personalizing recommendations for new or unauthenticated users in Airbnb’s cold-start scenarios, particularly under stringent privacy regulations that prohibit the use of persistent individual identifiers. To overcome this limitation, the authors propose a geolocation-based adaptive clustering method that dynamically aggregates approximately 1,000 proximate users into cohorts, thereby constructing group-level feature signals without relying on persistent user identities. This approach introduces a novel privacy-compliant mechanism driven by geographic proximity to enable effective personalization while preserving user anonymity. Online A/B experiments demonstrate that the proposed method significantly improves booking conversion rates, with especially pronounced gains for users exhibiting no prior activity or outdated behavioral data. The solution has been successfully deployed in production systems, including marketing landing pages and destination recommendation modules.

0 citationsRead paper

Understanding Guest Preferences and Optimizing Two-sided Marketplaces: Airbnb as an Example

Jun 30, 2026

This study investigates preference heterogeneity among users in two-sided markets with respect to rental listing prices and other attributes, aiming to enhance supply-demand matching and dynamic pricing mechanisms. By integrating economic modeling with causal inference methods, the research identifies systematic differences in decision-making behavior across user segments and leverages these insights to develop personalized recommendation systems and landlord pricing tools. The approach not only uncovers the structural sources of preference heterogeneity but also provides platforms with actionable strategies to optimize market operations. Empirical results demonstrate substantial improvements in matching efficiency, pricing accuracy, and user experience, thereby strengthening overall market equilibrium.

0 citationsRead paper