Institution profile

Hong Kong University of Science and Technology (Guangzhou)

Academic institutionasia · cn
Official website
Research library861linked papers
Opportunities0open roles
Selected work

Representative Papers

MAD: Multi-Alignment MEG-to-Text Decoding

Jun 03, 2024arXiv.org

Current non-invasive BCI-based language decoding faces three key bottlenecks: underutilization of magnetoencephalography (MEG) signals, poor cross-sentence generalization, and absence of multimodal fusion. Method: We propose the first end-to-end, multi-aligned MEG-to-text framework for natural language reconstruction from entirely unseen sentences. Our approach introduces a Transformer-based architecture that jointly aligns neural time series, phonemes, and semantics, integrating self-supervised pretraining with cross-modal contrastive learning to systematically unify speech, semantic, and dynamic temporal information. Results: On the Gwilliams dataset, our method achieves a BLEU-1 score of 10.44—improving by 4.95 (+93%) over the strongest baseline—demonstrating substantially enhanced open-vocabulary text generation capability. This work breaks critical limitations in generalizability and multimodal integration for non-invasive brain–computer interface–based language reconstruction.

18 citations3 influentialRead paper

CodeScaler: Scaling Code LLM Training and Test-Time Inference via Execution-Free Reward Models

Feb 04, 2026arXiv.org

This work addresses the limitations of existing reinforcement learning approaches for code large language models that rely on unit tests, which are often scarce and unreliable, hindering scalable training and inference. To overcome this, the authors propose CodeScaler—a test-case-free reward model for code generation—leveraging verified preference data, syntax-aware code extraction, and validity-preserving reward shaping to enable efficient scaling. Evaluated across four code benchmarks, CodeScaler outperforms execution-feedback-based RL by up to 4.23 points. Notably, when scaled to 44K problems without any test cases, it achieves a 14.64-point improvement while reducing inference latency by an order of magnitude. Furthermore, it surpasses prior methods by 3.3 points on coding tasks and by an average of 2.7 points across general and reasoning tasks in RM-Bench.

3 citationsRead paper

Vision-Language Models Meet Meteorology: Developing Models for Extreme Weather Events Detection with Heatmaps

Jun 14, 2024arXiv.org

Existing vision-language models (VLMs) exhibit color perception bias and imprecise spatial localization when interpreting meteorological heatmaps, leading to unreliable explanations for extreme weather event detection (EWED). To address this, we formulate EWED as a vision-language question answering (VQA) task and introduce three key contributions: (1) ClimateIQA—the first domain-specific VQA dataset for meteorology; (2) SPOT, a novel algorithm that enhances precise localization of heatmap color boundaries and critical regions; and (3) Climate-Zoo, a family of meteorology-specialized VLMs. Experiments demonstrate that our approach elevates EWED accuracy from 0% to over 90%, substantially outperforming general-purpose VLMs. All datasets, source code, and pretrained models are publicly released, establishing a reproducible benchmark and foundational infrastructure for AI-driven meteorology.

3 citationsRead paper

LangForce: Bayesian Decomposition of Vision Language Action Models via Latent Action Queries

Jan 21, 2026

This work addresses the limited generalization of current vision-language-action (VLA) models to novel instructions or multi-task settings, which stems from a collapse in the conditional mutual information between language instructions and actions—caused by the redundancy of instructions in training data where actions can be predicted directly from visual inputs alone. To tackle this “information collapse,” the paper formally characterizes the problem and introduces a Bayesian decomposition–based dual-branch architecture. This framework employs learnable latent action queries to separately model a vision-driven prior and a language-conditioned posterior, while explicitly optimizing pointwise mutual information (PMI) between actions and instructions to enforce instruction adherence. Evaluated on SimplerEnv and RoboCasa benchmarks, the method achieves significant generalization gains without additional data, improving out-of-distribution accuracy by 11.3% on SimplerEnv.

2 citationsRead paper

Direct Learning of Mesh and Appearance via 3D Gaussian Splatting

May 11, 2024arXiv.org

Existing 3D reconstruction methods suffer from low efficiency and limited quality due to indirect geometric learning and coupled geometry-appearance modeling. To address this, we propose the first end-to-end jointly optimized framework integrating explicit triangular meshes with 3D Gaussian points: differentiable 3D Gaussians are rigidly bound to mesh faces and jointly optimized under photometric supervision to reconstruct both geometry and surface appearance. Our approach breaks the conventional paradigm of decoupled geometry and appearance modeling, enabling efficient, high-fidelity reconstruction and real-time rendering. Quantitatively, it achieves a +1.8 dB PSNR improvement over prior methods on the DTU and BlendedMVS benchmarks. Moreover, the framework supports interactive mesh editing and incremental updates for dynamic scenes, significantly enhancing reconstruction efficiency and editing flexibility.

2 citationsRead paper
Recent publications

Latest Papers