SpecMER: Fast Protein Generation with K-mer Guided Speculative Decoding

2025-09-25

AI Summary

Autoregressive protein generation models suffer from high inference latency, poor biological plausibility due to the absence of structural/functional priors, and likelihood distribution shift—hindering high-throughput screening applications. To address these limitations, we propose SpecMER, a k-mer-guided speculative decoding framework. SpecMER constructs a lightweight draft model from conserved k-mer motifs extracted via multiple sequence alignment, enabling parallel candidate sequence scoring and refinement in collaboration with a large target model. This explicitly incorporates biological priors into the decoding process. Experiments demonstrate that SpecMER achieves 24–32% inference speedup over standard autoregressive decoding while preserving generation quality; it also significantly improves token acceptance rate and log-likelihood scores. By unifying computational efficiency with biologically informed constraints, SpecMER establishes a new paradigm for efficient and trustworthy de novo protein design.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Autoregressive models have transformed protein engineering by enabling the generation of novel protein sequences beyond those found in nature. However, their sequential inference introduces significant latency, limiting their utility in high-throughput protein screening. Speculative decoding accelerates generation by employing a lightweight draft model to sample tokens, which a larger target model then verifies and refines. Yet, in protein sequence generation, draft models are typically agnostic to the structural and functional constraints of the target protein, leading to biologically implausible outputs and a shift in the likelihood distribution of generated sequences. We introduce SpecMER (Speculative Decoding via k-mer Guidance), a novel framework that incorporates biological, structural, and functional priors using k-mer motifs extracted from multiple sequence alignments. By scoring candidate sequences in parallel and selecting those most consistent with known biological patterns, SpecMER significantly improves sequence plausibility while retaining the efficiency of speculative decoding. SpecMER achieves 24-32% speedup over standard autoregressive decoding, along with higher acceptance rates and improved sequence likelihoods.

Problem

• Accelerates protein sequence generation to reduce inference latency
• Improves biological plausibility by incorporating structural constraints
• Maintains efficiency while enhancing sequence acceptance rates

Innovation

• Uses k-mer motifs from sequence alignments
• Scores candidate sequences for biological plausibility
• Retains speculative decoding efficiency with higher acceptance