ShortListing Model: A Streamlined SimplexDiffusion for Discrete Variable Generation

2025-08-24

AI Summary

Discrete variable generation poses significant challenges in natural language processing and biomolecular sequence design, including difficulties in modeling discrete distributions and inefficient sampling. This paper introduces the Simplex-Centered Shortlist Model (SLM), which maps discrete sequences into a continuous simplex space and integrates discrete diffusion modeling with progressive candidate set pruning to dynamically shrink the search space during denoising. A novel classifier-free guidance mechanism is incorporated to enhance both unconditional generation quality and controllability. Experiments demonstrate that SLM achieves state-of-the-art performance on diverse tasks—including DNA regulatory sequence generation, protein sequence design, and multi-granularity language modeling—while exhibiting high efficiency, scalability, and cross-domain generalization. By unifying discrete structure modeling with geometrically informed diffusion and adaptive search, SLM establishes a new paradigm for generative modeling of discrete sequences.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Natural Language Processing: Generation
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Generative modeling of discrete variables is challenging yet crucial for applications in natural language processing and biological sequence design. We introduce the Shortlisting Model (SLM), a novel simplex-based diffusion model inspired by progressive candidate pruning. SLM operates on simplex centroids, reducing generation complexity and enhancing scalability. Additionally, SLM incorporates a flexible implementation of classifier-free guidance, enhancing unconditional generation performance. Extensive experiments on DNA promoter and enhancer design, protein design, character-level and large-vocabulary language modeling demonstrate the competitive performance and strong potential of SLM. Our code can be found at https://github.com/GenSI-THUAIR/SLM

Problem

• Develops simplex diffusion model for discrete variable generation
• Addresses complexity in natural language and biological sequences
• Enhances scalability and performance in unconditional generation

Innovation

• Simplex-based diffusion model for discrete variables
• Classifier-free guidance for unconditional generation
• Centroid-based approach reducing complexity and enhancing scalability