ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling

2026-06-23

AI Summary

This work addresses the challenge of modeling billions of low-activity users whose sparse interactions hinder effective representation learning. To overcome this, the authors propose a novel paradigm that efficiently transfers structured reasoning capabilities from a small-scale language model by constructing a typed Tree-of-Thought (ToT) to generate structured user state chains, enhanced with entropy constraints to promote reasoning diversity. A lightweight student model is then trained via supervised fine-tuning (SFT) and a newly introduced Outcome-Driven Segment-Aware Implicit Reward Policy Optimization (OSIPO) method to generalize across the entire user base. Remarkably, the approach requires offline inference for only 7.32% of users and achieves a statistically significant 6.738% improvement in LT30 in online A/B tests, substantially reducing computational overhead.

Abstract

Accurate user modeling often depends on rich interaction histories, which are unavailable for billions of low-activity users. Large Language Models (LLMs) can infer latent user states from static profiles, but this reasoning becomes unreliable when profiles are sparse, and applying an LLM to billions of users is prohibitively expensive. We present ScaleToT, which learns structured reasoning from a small LLM-processed subset and extends it to the broader low-activity user population. To improve reasoning reliability, ScaleToT constructs typed user-state chains with a bounded entropy-guided Tree-of-Thought (ToT) refinement procedure. To make this structured reasoning usable from sparse profiles, the teacher-curated chains are used to train a student model on static profiles through supervised fine-tuning (SFT) and Outcome-Driven Segment-Aware Implicit Reward Policy Optimization (OSIPO). ScaleToT then transfers the student's reasoning representations to a lightweight profile encoder, providing shared reasoning signals for the remaining users without LLM inference. We evaluate ScaleToT on lifetime value (LTV) prediction in a billion-scale advertising deployment. A randomized online A/B test increased LT30 by 6.738\%, while offline reasoning covered only 7.32\% of the potential population, greatly reducing compute cost compared with full-population reasoning.

Problem

• low-activity user modeling
• Large Language Models
• user profiling
• scalable reasoning
• sparse data

Innovation

• Tree-of-Thought
• structured reasoning
• low-activity user modeling
• supervised fine-tuning
• implicit reward optimization