Wisteria: A Unified Multi-Scale Feature Learning Framework for DNA Language Model

2026-05-07

AI Summary

Existing DNA language models struggle to jointly capture local motifs and global dependencies, limiting their ability to interpret the multi-scale regulatory semantics of genomes. This work proposes a unified multi-scale framework based on the Mamba architecture, which innovatively integrates gated dilated convolutions, gated MLPs, and Fourier attention mechanisms to simultaneously model local sequence patterns, long-range dependencies, and frequency-domain generalization, while supporting sequence length extrapolation. Evaluated across four benchmark settings encompassing both short- and long-range genomic dependencies, the proposed method substantially outperforms current DNA language models and achieves state-of-the-art performance on multiple downstream tasks.

Abstract

DNA language model aims to decipher the regulatory grammar and semantic of genomes by capturing long range dependencies in DNA sequences. Existing methods emphasize long range token interactions but often ignore the interplay between local motifs and global dependencies. In this paper, we propose Wisteria, a genomic language model that integrates multi scale feature learning within a unified framework for DNA sequence. Specifically, Wisteria augments the Mamba based architecture with gated dilated convolutions to capture local motifs and regulatory patterns, while gated multilayer perceptrons refine global dependencies. We further introduce a Fourier based attention mechanism to support frequency domain modeling, periodic extension and length generalization. Across four experimental settings with both short and long range dependencies, Wisteria demonstrates strong performance on downstream benchmarks against competitive DNA language model baselines. These results indicate that Wisteria effectively unifies local and global dependency modeling for multi scale genomic sequence analysis.

Problem

• DNA language model
• multi-scale feature learning
• local motifs
• global dependencies
• genomic sequence analysis

Innovation

• multi-scale feature learning
• gated dilated convolution
• Fourier-based attention
• DNA language model
• Mamba architecture