Selected work

Representative Papers

LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models

Nov 01, 2024arXiv.org

Mixture-of-Experts (MoE) architectures have become a key scaling paradigm for large language models (e.g., DeepSeek-V3, Llama-4), yet their prohibitive training and evaluation costs hinder systematic research. Method: We introduce LibMoE—the first open-source benchmark library for LLM-MoE—designed modularly, optimized for efficiency, and supporting comprehensive evaluation. Built on PyTorch, it implements Top-K routing, gradient sparsity, distributed optimization, and multi-dimensional metrics (e.g., accuracy, throughput, load balancing). Contribution/Results: LibMoE enables the first unified zero-shot benchmarking of five state-of-the-art MoE algorithms across three LLM families and eleven datasets. Our evaluation reveals convergent cross-task performance among leading methods. By significantly lowering entry barriers, LibMoE enhances reproducibility, extensibility, and standardization in MoE research.

4 citationsRead paper

Revisiting Kernel Attention with Correlated Gaussian Process Representation

Feb 27, 2025

This work addresses the limitation in Transformer-based uncertainty calibration—specifically, the restrictive symmetric kernel assumption of Gaussian processes (GPs) in existing GP-Transformer models. We propose a Correlated Gaussian Process (CGP) attention mechanism. Methodologically, we formulate self-attention as the cross-covariance between two correlated yet asymmetric GPs, thereby relaxing the symmetry constraint inherent in conventional GP-Transformers and enhancing representational capacity. To ensure scalability, we further introduce a sparse variational CGP approximation. Empirical evaluation across multiple benchmark tasks demonstrates that our approach consistently outperforms state-of-the-art GP-based Transformers, validating substantial improvements in uncertainty calibration accuracy, modeling flexibility, and predictive performance.

2 citationsRead paper

Distance-Based Tree-Sliced Wasserstein Distance

Mar 14, 2025

Existing Tree-Sliced Wasserstein on Systems of Lines (TSW-SL) methods rely solely on support point locations while ignoring the projection domain, and their fixed partitioning mappings lack Euclidean invariance, hindering preservation of high-dimensional topological structure. Method: We propose Distance-aware tree-sliced Wasserstein (Db-TSW), which introduces a class of distance-sensitive generalized splitting mappings that explicitly encode full positional information of the input metric, ensuring Euclidean invariance. We establish a Radon-transform-based theoretical framework, proving Db-TSW’s injectivity and metric validity, and design GPU-efficient tree sampling and distance computation mechanisms. Contribution/Results: Experiments demonstrate that Db-TSW significantly outperforms mainstream Sliced Wasserstein variants across diverse tasks—achieving notable accuracy gains—while maintaining linear time complexity and low computational overhead.

1 citationsRead paper

Voxel-based 3D Facies Segmentation from Seismic Data: A Comparative Study

Aug 14, 2026

This study addresses the limitations of 2D methods in seismic facies segmentation, which disrupt 3D spatial continuity and lack unified benchmarks. To overcome these issues, we construct the first open, standardized voxel-based 3D segmentation benchmark and evaluation framework. By employing a voxel-wise 3D architecture, rigorous data partitioning, and multidimensional metrics, this work systematically evaluates mainstream models and establishes strong baselines. The proposed approach effectively mitigates slice discontinuity artifacts while revealing both the potential and challenges inherent in 3D methodologies. Consequently, this research provides a reproducible, standardized evaluation framework and critical reference for geological pattern recognition, facilitating more robust comparative studies in the field.

0 citationsRead paper

ISRS-DETR: Detection-Guided Click Propagation for Remote Sensing Interactive Segmentation

Aug 03, 2026

This work addresses the limitation of existing interactive segmentation methods for remote sensing imagery, which treat all instances of the same class as a single object, preventing click sharing across distinct instances and necessitating excessive user interactions. To overcome this, the authors propose a detection-guided interactive segmentation framework that, for the first time, incorporates object-level detection cues during both training and inference. By leveraging strong inter-instance correlations among same-class objects, the method integrates an RF-DETR decoder with the segmentation backbone and introduces a dynamic Top-K click selection mechanism to convert high-confidence detection proposals into simulated clicks. This enables a single user click to effectively segment multiple instances of the same class. Evaluated on three remote sensing benchmarks, the approach achieves substantial gains in accuracy while significantly reducing the number of clicks per image (NoC-I), thereby enhancing practical usability.

0 citationsRead paper

Recent publications

Latest Papers

Voxel-based 3D Facies Segmentation from Seismic Data: A Comparative Study

Aug 14, 2026

This study addresses the limitations of 2D methods in seismic facies segmentation, which disrupt 3D spatial continuity and lack unified benchmarks. To overcome these issues, we construct the first open, standardized voxel-based 3D segmentation benchmark and evaluation framework. By employing a voxel-wise 3D architecture, rigorous data partitioning, and multidimensional metrics, this work systematically evaluates mainstream models and establishes strong baselines. The proposed approach effectively mitigates slice discontinuity artifacts while revealing both the potential and challenges inherent in 3D methodologies. Consequently, this research provides a reproducible, standardized evaluation framework and critical reference for geological pattern recognition, facilitating more robust comparative studies in the field.

0 citationsRead paper

ISRS-DETR: Detection-Guided Click Propagation for Remote Sensing Interactive Segmentation

Aug 03, 2026

This work addresses the limitation of existing interactive segmentation methods for remote sensing imagery, which treat all instances of the same class as a single object, preventing click sharing across distinct instances and necessitating excessive user interactions. To overcome this, the authors propose a detection-guided interactive segmentation framework that, for the first time, incorporates object-level detection cues during both training and inference. By leveraging strong inter-instance correlations among same-class objects, the method integrates an RF-DETR decoder with the segmentation backbone and introduces a dynamic Top-K click selection mechanism to convert high-confidence detection proposals into simulated clicks. This enables a single user click to effectively segment multiple instances of the same class. Evaluated on three remote sensing benchmarks, the approach achieves substantial gains in accuracy while significantly reducing the number of clicks per image (NoC-I), thereby enhancing practical usability.

0 citationsRead paper

Precise Video-to-Audio Generation with Cross-Modal Alignment in Latent Space

Jul 07, 2026

This work addresses key limitations in existing video-to-audio generation methods—namely, high computational costs from multi-stage training, temporal detail degradation due to reliance on textual intermediaries, and the absence of sound-centric semantic annotations. To overcome these challenges, we propose Flowley, an end-to-end, single-stage audio-visual generation architecture that introduces a novel progressive soft-masked cross-modal attention mechanism, enabling synchronized audio-visual embedding without additional computational overhead. Additionally, we design SoundCap, a plug-and-play sound-aware captioning pipeline that constructs descriptive sound semantics. Experimental results demonstrate that Flowley achieves state-of-the-art performance on VGGSound, and when integrated with SoundCap, it surpasses the strongest closed-source baselines in zero-shot audio generation quality.

0 citationsRead paper

Teaching Code LLMs to Reason with Intermediate Formal Specifications

Jul 05, 2026

Existing large code models struggle to generate executable intermediate formal specifications, limiting precise verification and repair of program behavioral errors. This work proposes SpecCoder, a novel framework that focuses on generating executable inline assertions at critical program locations, thereby transforming static annotations into verifiable evidence. SpecCoder employs verification-guided training, fine-tuning the Qwen2.5-Coder series models using correct programs, behavioral mutants, and multi-round specification refinement trajectories. Evaluated on the HumanExec benchmark, SpecCoder substantially improves the correctness (+55.8%), completeness (+358.1%), and assertion validity (+26.6%) of inline specifications, significantly enhancing program verification and repair capabilities.

0 citationsRead paper

Benchmarking Code Improvement with Progressive, Adaptive, and Interactive Feedback

Jul 01, 2026

Traditional evaluation of code generation and repair relies solely on binary correctness, overlooking critical dimensions such as incremental improvement, feedback utilization, and behavioral preservation during the repair process. To address this limitation, this work proposes PAIR-Bench, a novel benchmark that guides large language models through feedback-driven iterative repair via a structured feedback protocol combining failure-region control and prompt-depth control. The benchmark introduces progressive prompting, grouping of failure scenarios, and multi-level repair guidance, alongside trajectory-level evaluation metrics. For the first time, it enables fine-grained, adaptive assessment of the code improvement process, outperforming conventional coarse-grained evaluation paradigms in terms of target repair efficacy, generalization capability, behavioral consistency, and required human assistance.

0 citationsRead paper