Institution profile

Shanghai Jiao Tong University

Academic institutionasia · cn
Official website
Research library5,781linked papers
Opportunities0open roles
Selected work

Representative Papers

Making Images Real Again: A Comprehensive Survey on Deep Image Composition

Jun 28, 2021arXiv.org

This work addresses the pervasive visual inconsistency—arising from mismatches in color, scale, shape, illumination, shadows, and reflections—between inserted objects and background scenes in image compositing. To this end, we propose the first holistic taxonomy and unified framework for image synthesis, encompassing core subtasks including object placement, scene fusion, color harmonization, and shadow/reflection generation. Methodologically, the framework integrates CNNs, GANs, diffusion models, and multi-scale feature alignment techniques. Our contributions are threefold: (1) a standardized benchmark suite unifying major datasets (e.g., iHarmony4, HCOCO); (2) libcom—an open-source, modular image compositing toolbox implementing over ten state-of-the-art algorithms; and (3) a paradigm shift toward systematic modeling and engineering-ready deployment in image synthesis. Extensive experiments demonstrate the framework’s generality, robustness, and practical utility across diverse compositional scenarios.

72 citations2 influentialRead paper

Derail Yourself: Multi-turn LLM Jailbreak Attack through Self-discovered Clues

Oct 14, 2024arXiv.org

This paper identifies a novel security vulnerability in large language models (LLMs): in multi-turn dialogues, LLMs exhibit heightened sensitivity to “natural distribution shifts”—semantically related yet superficially benign prompts—that evade existing alignment mechanisms. To exploit this, we propose ActorAttack, the first multi-turn jailbreaking framework grounded in actor-network theory; it leverages LLM-driven role-association modeling to automatically discover covert, diverse attack paths while preserving intent invisibility. Our contributions are threefold: (1) we introduce SafeMTData, the first open-source multi-turn adversarial dataset; (2) ActorAttack significantly outperforms state-of-the-art single- and multi-turn baselines across aligned models including GPT-4o, Claude, and Qwen; and (3) fine-tuning on SafeMTData substantially enhances model robustness against multi-turn attacks.

26 citations7 influentialRead paper

HeteroLLM: Accelerating Large Language Model Inference on Mobile SoCs platform with Heterogeneous AI Accelerators

Jan 11, 2025arXiv.org

To address the lack of systematic performance modeling and efficient scheduling for collaborative large language model (LLM) inference across heterogeneous AI accelerators (e.g., GPU, NPU) in mobile SoCs, this paper proposes the first fine-grained heterogeneous execution framework tailored for mobile unified memory architectures. The framework introduces a novel hierarchical and tensor-level dynamic scheduling mechanism that operates separately during the prefill and decoding phases. It incorporates phase-aware dynamic tensor partitioning strategies and lightweight synchronization primitives to fully exploit the unified virtual address space while minimizing cross-device synchronization overhead. Experimental evaluation demonstrates substantial acceleration in mobile LLM inference: our framework achieves 9.99× and 4.36× speedups over MLC and MNN, respectively, establishing it as the fastest known mobile LLM inference engine to date.

8 citations2 influentialRead paper

DAG-Plan: Generating Directed Acyclic Dependency Graphs for Dual-Arm Cooperative Planning

Jun 14, 2024arXiv.org

Long-horizon collaborative tasks for dual robotic arms face challenges including complex spatiotemporal dependencies among subtasks, difficulty in dynamic action allocation, and limited expressiveness of linear programming formulations. This paper proposes the first LLM-driven DAG-structured task decomposition framework, which automatically parses high-level instructions into directed acyclic graphs (DAGs) encoding dependency constraints, and integrates environment perception to enable real-time, dynamic action allocation and parallel adaptive execution across both arms. The method breaks away from predefined operational paradigms, supporting end-to-end, interpretable, and generalizable collaborative planning. Evaluated on the Dual-Arm Kitchen benchmark, it achieves a 52.8% efficiency gain over single-arm systems, improves success rate by 48% and reduces LLM query count by 84.1% compared to conventional dual-arm planners, significantly enhancing robustness and scalability in complex scenarios.

8 citationsRead paper

d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation

Jan 12, 2026

This work addresses the trade-off between generation accuracy and parallel efficiency in deployed diffusion-based large language models (dLLMs). To reconcile this tension, we propose a pseudo-trajectory distillation training strategy that guides the model to predict high-confidence tokens early in the denoising process. This approach is integrated with entropy-based multi-block parallel decoding and a dynamic KV cache refresh mechanism, enabling substantial acceleration of inference while preserving output quality. We further introduce a unified evaluation metric, Area Under the Parallelism–Accuracy curve (AUP), to jointly assess accuracy and parallelism. Experimental results demonstrate that our method achieves up to a 10× speedup over baseline dLLMs and a 5× speedup compared to autoregressive models, with negligible degradation in generation accuracy.

7 citations1 influentialRead paper
Recent publications

Latest Papers