Microsoft · Agora

Selected work

Representative Papers

Semantic Image Synthesis via Diffusion Models

Jun 30, 2022arXiv.org

Existing GAN-based semantic image synthesis methods suffer from inherent trade-offs between generation quality and diversity. To address this, we propose the first semantic image synthesis framework built upon denoising diffusion probabilistic models (DDPMs). Our method fundamentally decouples two key inputs: noisy images are fed into the U-Net encoder, while semantic layouts guide a dedicated decoder path via multi-level Spatially-Adaptive Denormalization (SPADE). Crucially, we introduce classifier-free guidance—the first such application in semantic diffusion synthesis—to substantially improve layout-to-pixel alignment. Evaluated on four standard benchmarks—Cityscapes, ADE20K, COCO-Stuff, and Mapillary Vistas—our approach achieves state-of-the-art performance: FID of 14.3 and LPIPS of 0.52, demonstrating significant gains in both visual fidelity and semantic consistency.

170 citations26 influentialRead paper

Knowledge Distillation of Large Language Models

Jun 14, 2023International Conference on Learning Representations

To address distribution misalignment in generative tasks, overfitting in low-probability regions, and severe exposure bias in knowledge distillation for open-source white-box large language models (LLMs), this work introduces reverse KL divergence—first applied to LLM distillation—to formulate a generation-aware optimization objective and develop an efficient solver. We further propose an instruction-tuning distillation framework coupled with multi-scale model adaptation, enabling effective distillation across model sizes from 120M to 13B parameters. Experiments demonstrate substantial improvements in output calibration, long-text coherence, and instruction-following capability, consistently outperforming state-of-the-art baselines across response accuracy, overall quality, and other key metrics. All code, datasets, and distilled models are publicly released.

92 citations11 influentialRead paper

Budget Pacing in Repeated Auctions: Regret and Efficiency without Convergence

May 18, 2022Information Technology Convergence and Services

This paper investigates the impact of dynamic bidding pacing algorithms on group liquid welfare and individual dynamic regret in repeated ad auctions under budget constraints. To overcome the limitation of prior work—reliance on convergence assumptions about algorithmic dynamics—we propose a novel theoretical framework that makes no such assumptions. First, we establish that liquid welfare is guaranteed to be at least 50% of the optimal expected value, irrespective of convergence. Second, we derive an upper bound on dynamic regret tailored to time-varying budgets. Third, we design a gradient-based linear pacing algorithm within the core auction framework, integrating monotonic return-on-spend modeling and dynamic regret analysis to ensure broad applicability across first-price, second-price, and generalized second-price auctions. Empirical validation on Bing Ads data confirms the theoretical guarantees.

36 citations1 influentialRead paper

Hey, That's My Model! Introducing Chain & Hash, An LLM Fingerprinting Technique

Jul 15, 2024arXiv.org

To address risks of large language model (LLM) theft and misuse, this paper proposes a verifiable, tamper-resistant model fingerprinting technique. Methodologically, it constructs a cryptographic hash chain using question-answer pairs and SHA-256, enforcing fine-grained response constraints and hash binding to ensure strong integrity verification. It is the first work to formally define and satisfy five core fingerprint properties: transparency, efficiency, persistence, robustness, and unforgeability. Extensive experiments across multiple LLMs demonstrate that the fingerprint withstands benign modifications—including fine-tuning and pruning—as well as adversarial erasure attacks, while preserving near-original model performance post-embedding. This work delivers the first complete solution for LLM copyright protection and provenance tracking that simultaneously achieves theoretical rigor and engineering practicality.

12 citations1 influentialRead paper

Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulation

Dec 17, 2023

This work addresses the semantic gap between natural language instructions and robotic physical actions to enhance the naturalness and reliability of human-robot collaboration. We propose the first four-dimensional taxonomy for language-conditioned robotic manipulation—comprising reward shaping, policy learning, neurosymbolic AI, and foundation model–driven approaches—and systematically analyze their fundamental limitations in generalization and safety. Integrating large language models (LLMs), vision-language models (VLMs), neurosymbolic reasoning, and multimodal semantic parsing, we develop a unified analytical framework spanning semantic extraction, environmental assessment, and auxiliary task design. Our analysis rigorously characterizes the performance boundaries of each paradigm for the first time, establishing theoretical foundations and concrete technical pathways toward safe, generalizable, and interpretable language-driven robotic systems.

10 citationsRead paper

Recent publications