MORES: Mobile Reasoning-as-a-Service via Distributed LLM Inference-Time Scaling

2026-07-09

AI Summary

Deploying large language models on resource-constrained edge devices entails substantial computational and memory overhead. This work proposes MORES, a mobile inference-as-a-service framework that, for the first time, distributes the implicit recursive structure of inference across edge and cloud resources. By integrating Semantic Mixture-of-Experts (Semantic MoE) with deep reinforcement learning, MORES enables adaptive scheduling of computation and communication resources tailored to wireless heterogeneous environments. The proposed approach significantly enhances system efficiency, achieving approximately 18% higher throughput than a Soft Actor-Critic baseline under identical conditions, while allowing edge devices to dynamically leverage cloud-based inference capabilities on demand.

Abstract

Inference-time scaling has emerged as an effective approach for enhancing the capabilities of Large Language Models (LLMs), addressing the growing demand for stronger reasoning without increasing model size. This novel form of LLM scaling comprises two representative approaches: explicit reasoning, which generates intermediate chain-of-thought tokens during an explicit thinking phase, and implicit reasoning, which iteratively updates hidden states in the latent space without producing explicit outputs. Despite their effectiveness, both paradigms incur substantial computational and memory overhead, raising challenges for deployment on resource-constrained edge devices. To address these issues, we propose a Mobile Reasoning-as-aService (MORES) framework that treats reasoning as a computational service accessible to edge devices over wireless networks. Focusing on implicit reasoning, we leverage its recursive structure to partition hiddenstate updates between edge devices and servers, enabling cooperative inference that allows devices to access additional cloud computation on demand. To optimize long-term performance, we formulate a joint computation and communication scheduling problem and solve it using a semantic Mixture-of-Experts (MoE)-based Deep Reinforcement Learning (DRL) algorithm to address heterogeneity in wireless conditions and task demands. The agent adaptively allocates resources by adjusting the number of recurrent steps and the transmission pruning rate, while a semantic router enables high-speed gating for real-time expert selection. Experimental results show that the proposed method achieves an approximately 18% improvement in system throughput over the baseline Soft Actor-Critic (SAC) algorithm. Our code is available at https://github.com/NICE-HKU/MORES.

Problem

• LLM inference-time scaling
• implicit reasoning
• edge computing
• resource-constrained devices
• distributed inference

Innovation

• Implicit Reasoning
• Distributed Inference
• Semantic Mixture-of-Experts
• Deep Reinforcement Learning
• Edge-Cloud Collaboration