Selected work

Representative Papers

ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution

Jul 29, 2026

This work addresses the limited flexibility of existing EEG denoising and super-resolution methods with respect to sequence length, number of channels, electrode placement, and temporal segments. To overcome these constraints, the authors propose ZUNA1.1, a 380-million-parameter diffusion autoencoder that, for the first time, enables unified reconstruction of EEG signals under arbitrary missing patterns across temporal, channel, and spatial dimensions. The model supports variable-length inputs (up to 30 seconds), arbitrary numbers of channels, flexible electrode configurations, and can recover data from any temporal segment within a channel. Experimental results demonstrate that ZUNA1.1 matches or exceeds the performance of ZUNA1 across diverse reconstruction tasks and significantly outperforms conventional approaches such as spherical spline interpolation, exhibiting exceptional capabilities in both denoising and super-resolution. The code is publicly available under the Apache 2.0 license.

0 citationsRead paper

ZONOS2 Technical Report

Jun 23, 2026

This work proposes a high-performance text-to-speech (TTS) system based on a sparse mixture-of-experts (MoE) architecture to address challenges in naturalness, prosody, and voice cloning fidelity. By constructing an 8-billion-parameter MoE model—of which only 900 million parameters are activated per inference—and training it on 6 million hours of speech data, the approach substantially increases model capacity while maintaining manageable computational costs through optimized conditioning and inference pipelines. The resulting system achieves state-of-the-art performance in naturalness, speaker similarity, word error rate, and on the newly introduced ZTTS1-Eval benchmark, all while supporting low-latency streaming inference. The implementation has been open-sourced under the Apache 2.0 license.

0 citationsRead paper

Can Scale Save Us From Plasticity Loss in Large Language Models?

Jun 23, 2026

This study addresses the susceptibility of large language models to loss of plasticity during continual learning—specifically, their diminished capacity to effectively integrate new knowledge while retaining previously acquired information. By systematically training GPT-style Transformer models with non-embedding parameter counts ranging from 5M to 314M under both multilingual continual learning and static training regimes, the authors evaluate model plasticity using a Vietnamese probing task. They provide the first empirical evidence of plasticity loss in modern large language models, demonstrating that this phenomenon occurs not only under abrupt task shifts but also during static multilingual training. Although increasing model scale mitigates plasticity loss sublinearly, it does not prevent it entirely, indicating that even large models eventually lose the ability to adapt efficiently to new data after prolonged training.

0 citationsRead paper

Scaling Adaptive Depth with Norm-Agnostic Residual Networks

Jun 14, 2026

This work addresses the performance limitations of deep residual networks, which suffer from rapidly growing residual stream norms that suppress updates in deeper layers during scaling. To overcome this, the authors propose Norm-Agnostic Residual architecture (NAG), which decouples the magnitude and direction of residual streams to preserve effective layer contributions as depth increases. Additionally, they introduce a Mixture-of-Depth (MoD) mechanism that adaptively skips either attention or MLP layers. By treating depth sparsity as a novel scaling dimension under a fixed computational budget, the method achieves performance on par with full-depth baselines using only 20%–25% MoD rates, significantly reducing forward FLOPs and activation memory while enabling more efficient training of deeper models.

0 citationsRead paper

Zamba2-VL Technical Report

May 29, 2026

This work addresses the inefficiency of visual language models (VLMs) in tasks such as image understanding, reasoning, OCR, localization, and counting—particularly their high inference latency and limited deployability on edge devices. To this end, it introduces Zamba2, the first hybrid architecture for VLMs that integrates Mamba2 state-space layers with a small number of shared Transformer blocks. This design enables near-linear prefill computation and constant-size recurrent states, achieving approximately one order of magnitude lower first-token latency than existing state-space models and hybrid VLMs at 1.2B and 2.7B scales. Despite its compact size, the model matches the performance of mainstream Transformer-based VLMs across multiple visual-language benchmarks, substantially enhancing practicality and deployment efficiency on resource-constrained edge platforms.

0 citationsRead paper

Recent publications

Latest Papers

ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution

Jul 29, 2026

This work addresses the limited flexibility of existing EEG denoising and super-resolution methods with respect to sequence length, number of channels, electrode placement, and temporal segments. To overcome these constraints, the authors propose ZUNA1.1, a 380-million-parameter diffusion autoencoder that, for the first time, enables unified reconstruction of EEG signals under arbitrary missing patterns across temporal, channel, and spatial dimensions. The model supports variable-length inputs (up to 30 seconds), arbitrary numbers of channels, flexible electrode configurations, and can recover data from any temporal segment within a channel. Experimental results demonstrate that ZUNA1.1 matches or exceeds the performance of ZUNA1 across diverse reconstruction tasks and significantly outperforms conventional approaches such as spherical spline interpolation, exhibiting exceptional capabilities in both denoising and super-resolution. The code is publicly available under the Apache 2.0 license.

0 citationsRead paper

ZONOS2 Technical Report

Jun 23, 2026

This work proposes a high-performance text-to-speech (TTS) system based on a sparse mixture-of-experts (MoE) architecture to address challenges in naturalness, prosody, and voice cloning fidelity. By constructing an 8-billion-parameter MoE model—of which only 900 million parameters are activated per inference—and training it on 6 million hours of speech data, the approach substantially increases model capacity while maintaining manageable computational costs through optimized conditioning and inference pipelines. The resulting system achieves state-of-the-art performance in naturalness, speaker similarity, word error rate, and on the newly introduced ZTTS1-Eval benchmark, all while supporting low-latency streaming inference. The implementation has been open-sourced under the Apache 2.0 license.

0 citationsRead paper

Can Scale Save Us From Plasticity Loss in Large Language Models?

Jun 23, 2026

This study addresses the susceptibility of large language models to loss of plasticity during continual learning—specifically, their diminished capacity to effectively integrate new knowledge while retaining previously acquired information. By systematically training GPT-style Transformer models with non-embedding parameter counts ranging from 5M to 314M under both multilingual continual learning and static training regimes, the authors evaluate model plasticity using a Vietnamese probing task. They provide the first empirical evidence of plasticity loss in modern large language models, demonstrating that this phenomenon occurs not only under abrupt task shifts but also during static multilingual training. Although increasing model scale mitigates plasticity loss sublinearly, it does not prevent it entirely, indicating that even large models eventually lose the ability to adapt efficiently to new data after prolonged training.

0 citationsRead paper

Scaling Adaptive Depth with Norm-Agnostic Residual Networks

Jun 14, 2026

This work addresses the performance limitations of deep residual networks, which suffer from rapidly growing residual stream norms that suppress updates in deeper layers during scaling. To overcome this, the authors propose Norm-Agnostic Residual architecture (NAG), which decouples the magnitude and direction of residual streams to preserve effective layer contributions as depth increases. Additionally, they introduce a Mixture-of-Depth (MoD) mechanism that adaptively skips either attention or MLP layers. By treating depth sparsity as a novel scaling dimension under a fixed computational budget, the method achieves performance on par with full-depth baselines using only 20%–25% MoD rates, significantly reducing forward FLOPs and activation memory while enabling more efficient training of deeper models.

0 citationsRead paper

Zamba2-VL Technical Report

May 29, 2026

This work addresses the inefficiency of visual language models (VLMs) in tasks such as image understanding, reasoning, OCR, localization, and counting—particularly their high inference latency and limited deployability on edge devices. To this end, it introduces Zamba2, the first hybrid architecture for VLMs that integrates Mamba2 state-space layers with a small number of shared Transformer blocks. This design enables near-linear prefill computation and constant-size recurrent states, achieving approximately one order of magnitude lower first-token latency than existing state-space models and hybrid VLMs at 1.2B and 2.7B scales. Despite its compact size, the model matches the performance of mainstream Transformer-based VLMs across multiple visual-language benchmarks, substantially enhancing practicality and deployment efficiency on resource-constrained edge platforms.

0 citationsRead paper