🤖 AI Summary
Contemporary language models suffer from three key limitations in long-story generation: deteriorating coherence, uncontrolled endings, and poor length controllability. To address these, we propose CWC (Contextual Weight Calibrator), which dynamically gates short- and long-term contextual information, and LSP (Long-Story Positional encoding), a novel structural narrative position encoder—marking the first integration of structured narrative positioning with adaptive context gating. Built upon the Transformer architecture, our approach incorporates memory augmentation, discourse-level token modeling, multi-scale context weighting, and a three-stage training regimen on long-story data. Evaluated across multiple long-story benchmarks, our method significantly outperforms strong baselines—including PlotMachine—on coherence, completeness, relevance, and repetitiveness. Moreover, it enables zero-shot cross-length generalization. This work establishes a new paradigm for controllable, coherent, and naturally concluded long-text generation.
📝 Abstract
A human author can write any length of story without losing coherence. Also, they always bring the story to a proper ending, an ability that current language models lack. In this work, we present the LongStory for coherent, complete, and length-controlled long story generation. LongStory introduces two novel methodologies: (1) the long and short-term contexts weight calibrator (CWC) and (2) long story structural positions (LSP). The CWC adjusts weights for long-term context Memory and short-term context Cheating, acknowledging their distinct roles. The LSP employs discourse tokens to convey the structural positions of a long story. Trained on three datasets with varied average story lengths, LongStory outperforms other baselines, including the strong story generator Plotmachine, in coherence, completeness, relevance, and repetitiveness. We also perform zero-shot tests on each dataset to assess the model's ability to predict outcomes beyond its training data and validate our methodology by comparing its performance with variants of our model.
