Institution profile

Baseten

Industry researchnorthamerica · us
Official website
Research library5linked papers
Opportunities29open roles
Selected work

Representative Papers

Can a Language Model Learn Facts Continually in Its Weights?

Jul 12, 2026

This study investigates whether large language models can continuously learn and accumulate new factual knowledge through weight updates without catastrophically forgetting previously acquired information. Focusing on the Qwen3 model, the authors sequentially inject fictional facts via weight editing and systematically evaluate the impact of different training formulations on knowledge retention and usability using held-out question answering, KL divergence analysis, and context-based recovery tests. The findings reveal that training with diverse phrasings boosts knowledge retention to 46%, compared to merely 1% with a single statement. Moreover, ostensibly “forgotten” facts remain implicitly encoded in the weights and can be recovered to 77–80% accuracy with appropriate prompting. These results suggest that reliable knowledge transfer depends more on contextual activation than on static weight embedding, indicating that forgetting primarily stems from query redirection masking rather than actual knowledge erasure.

0 citationsRead paper

Still: Amortized KV Cache Compaction in a Single Forward Pass

Jun 05, 2026

This work addresses the severe memory bottleneck caused by KV caching in long-context inference, necessitating compression methods that simultaneously achieve light weight, high expressivity, and cross-trajectory reusability. The authors propose Still, a lightweight Perceiver module embedded at each layer of a frozen base model, which enables synthetic KV cache compression in a single forward pass without per-context optimization. Trained solely on top of frozen foundation models, Still is the first method to jointly satisfy all three desiderata—lightweight design, strong representational capacity, and reusability—enabling efficient inference and flexible summarization even under extreme compression ratios (8–200×). Evaluated on Qwen and Gemma with 128k context lengths, Still significantly outperforms existing approaches, achieving gains of 8–22 points over the strongest baseline on the RULER benchmark and setting new state-of-the-art results on LongBench summarization tasks.

0 citationsRead paper

RadixMLP -- Intra-batch Deduplication for Causal Transformers

Jan 21, 2026

This work addresses the significant redundancy in causal Transformer batch inference, where shared prefixes—such as system prompts or few-shot examples—are redundantly processed through repeated MLP activations. To mitigate this, the authors propose a stateless, prefix-Trie-based compressed inference method that constructs a Trie structure over shared input segments, enabling their unified representation in a single forward pass and restoring outputs only at attention boundaries. This approach uniquely integrates prefix deduplication with position-aware MLPs, LayerNorm, and embedding layers, achieving intra-batch redundancy elimination without caching or maintaining state. Compatible with standard Transformer architectures, the method yields end-to-end speedups of 1.44–1.59× on MS MARCO v1.1 using Qwen3 models (0.6B–8B parameters), with synthetic data demonstrating up to 5× acceleration.

0 citationsRead paper
Recent publications

Latest Papers

Can a Language Model Learn Facts Continually in Its Weights?

Jul 12, 2026

This study investigates whether large language models can continuously learn and accumulate new factual knowledge through weight updates without catastrophically forgetting previously acquired information. Focusing on the Qwen3 model, the authors sequentially inject fictional facts via weight editing and systematically evaluate the impact of different training formulations on knowledge retention and usability using held-out question answering, KL divergence analysis, and context-based recovery tests. The findings reveal that training with diverse phrasings boosts knowledge retention to 46%, compared to merely 1% with a single statement. Moreover, ostensibly “forgotten” facts remain implicitly encoded in the weights and can be recovered to 77–80% accuracy with appropriate prompting. These results suggest that reliable knowledge transfer depends more on contextual activation than on static weight embedding, indicating that forgetting primarily stems from query redirection masking rather than actual knowledge erasure.

0 citationsRead paper

Still: Amortized KV Cache Compaction in a Single Forward Pass

Jun 05, 2026

This work addresses the severe memory bottleneck caused by KV caching in long-context inference, necessitating compression methods that simultaneously achieve light weight, high expressivity, and cross-trajectory reusability. The authors propose Still, a lightweight Perceiver module embedded at each layer of a frozen base model, which enables synthetic KV cache compression in a single forward pass without per-context optimization. Trained solely on top of frozen foundation models, Still is the first method to jointly satisfy all three desiderata—lightweight design, strong representational capacity, and reusability—enabling efficient inference and flexible summarization even under extreme compression ratios (8–200×). Evaluated on Qwen and Gemma with 128k context lengths, Still significantly outperforms existing approaches, achieving gains of 8–22 points over the strongest baseline on the RULER benchmark and setting new state-of-the-art results on LongBench summarization tasks.

0 citationsRead paper

RadixMLP -- Intra-batch Deduplication for Causal Transformers

Jan 21, 2026

This work addresses the significant redundancy in causal Transformer batch inference, where shared prefixes—such as system prompts or few-shot examples—are redundantly processed through repeated MLP activations. To mitigate this, the authors propose a stateless, prefix-Trie-based compressed inference method that constructs a Trie structure over shared input segments, enabling their unified representation in a single forward pass and restoring outputs only at attention boundaries. This approach uniquely integrates prefix deduplication with position-aware MLPs, LayerNorm, and embedding layers, achieving intra-batch redundancy elimination without caching or maintaining state. Compatible with standard Transformer architectures, the method yields end-to-end speedups of 1.44–1.59× on MS MARCO v1.1 using Qwen3 models (0.6B–8B parameters), with synthetic data demonstrating up to 5× acceleration.

0 citationsRead paper