Selected work

Representative Papers

Assessing Sample Quality in Conditional Generation under Compositional Shift

Jun 08, 2026

This work addresses the challenge of evaluating conditional generation quality in compositional extrapolation settings, where the true target distribution is unavailable. The authors propose a post-hoc, instance-wise confidence scoring mechanism that requires no access to the target distribution. By constructing estimable metrics based on data manifold compatibility and attribute contrastive distance, the method holistically assesses both global realism and attribute fidelity. Notably, it incurs no additional training and is directly applicable to off-the-shelf pre-trained generative models. To the best of our knowledge, this is the first approach enabling effective evaluation of compositional extrapolation samples, facilitating sample filtering, ranking, and pre-generation abstention. Experiments on biological imaging and visual benchmarks demonstrate substantial improvements in morphological fidelity and downstream predictive performance, along with the capability for early abstention during generation.

0 citationsRead paper

Toward Identifiable Sparse Autoencoders

May 29, 2026

This work addresses the instability and high reconstruction error commonly observed in sparse autoencoders (SAEs), which stem from non-identifiability leading to inconsistent dictionaries and encodings during training. To overcome this limitation, the authors propose the identifiable Sparse Autoencoder (iSAE), built upon the TopK SAE architecture. By introducing structural modifications and a training strategy that enforces an approximate restricted isometry condition, iSAE achieves— for the first time in practice—approximately identifiable sparse codes. This advancement significantly enhances model stability and reduces reconstruction error, while also establishing a theoretical bridge between modern SAEs and classical dictionary learning frameworks.

0 citationsRead paper

Parallel Token Prediction for Language Models

Dec 24, 2025

Autoregressive decoding in large language models incurs high latency, and existing multi-token prediction methods rely on strong independence assumptions that limit modeling fidelity. Method: We propose Parallel Token Prediction (PTP), the first framework to internalize the sampling process into the model architecture, enabling joint generation of multiple semantically coherent tokens in a single Transformer forward pass while strictly preserving expressivity over any autoregressive distribution—thereby eliminating restrictive independence assumptions. PTP integrates inverse autoregressive training with explicit sampling modeling and supports both teacher-free and distillation-based training. Results: Evaluated on Vicuna-7B, PTP achieves 4.12 average accepted tokens per speculative step on Spec-Bench, maintains full modeling capability for long-sequence generation, and attains state-of-the-art performance in speculative decoding.

0 citationsRead paper

Recent publications

Latest Papers

Assessing Sample Quality in Conditional Generation under Compositional Shift

Jun 08, 2026

This work addresses the challenge of evaluating conditional generation quality in compositional extrapolation settings, where the true target distribution is unavailable. The authors propose a post-hoc, instance-wise confidence scoring mechanism that requires no access to the target distribution. By constructing estimable metrics based on data manifold compatibility and attribute contrastive distance, the method holistically assesses both global realism and attribute fidelity. Notably, it incurs no additional training and is directly applicable to off-the-shelf pre-trained generative models. To the best of our knowledge, this is the first approach enabling effective evaluation of compositional extrapolation samples, facilitating sample filtering, ranking, and pre-generation abstention. Experiments on biological imaging and visual benchmarks demonstrate substantial improvements in morphological fidelity and downstream predictive performance, along with the capability for early abstention during generation.

0 citationsRead paper

Toward Identifiable Sparse Autoencoders

May 29, 2026

This work addresses the instability and high reconstruction error commonly observed in sparse autoencoders (SAEs), which stem from non-identifiability leading to inconsistent dictionaries and encodings during training. To overcome this limitation, the authors propose the identifiable Sparse Autoencoder (iSAE), built upon the TopK SAE architecture. By introducing structural modifications and a training strategy that enforces an approximate restricted isometry condition, iSAE achieves— for the first time in practice—approximately identifiable sparse codes. This advancement significantly enhances model stability and reduces reconstruction error, while also establishing a theoretical bridge between modern SAEs and classical dictionary learning frameworks.

0 citationsRead paper

Parallel Token Prediction for Language Models

Dec 24, 2025

Autoregressive decoding in large language models incurs high latency, and existing multi-token prediction methods rely on strong independence assumptions that limit modeling fidelity. Method: We propose Parallel Token Prediction (PTP), the first framework to internalize the sampling process into the model architecture, enabling joint generation of multiple semantically coherent tokens in a single Transformer forward pass while strictly preserving expressivity over any autoregressive distribution—thereby eliminating restrictive independence assumptions. PTP integrates inverse autoregressive training with explicit sampling modeling and supports both teacher-free and distillation-based training. Results: Evaluated on Vicuna-7B, PTP achieves 4.12 average accepted tokens per speculative step on Spec-Bench, maintains full modeling capability for long-sequence generation, and attains state-of-the-art performance in speculative decoding.

0 citationsRead paper