Position: Avoid Overstretching LLMs for every Enterprise Task

2026-05-10

AI Summary

This work addresses the inefficiency and unreliability of directly deploying large language models (LLMs) or their distilled variants for enterprise tasks, which are typically deterministic, structured, and heavily reliant on domain-specific knowledge under strict constraints of cost, latency, and reliability. To overcome these limitations, the authors propose a modular AI architecture that confines LLMs to structured information extraction while offloading knowledge storage and reasoning to dedicated knowledge bases and symbolic systems. This design circumvents the bottlenecks of monolithic models in terms of interpretability, reliability, and maintainability. Theoretical analysis and system implementation demonstrate that the proposed architecture offers a more efficient, transparent, and sustainable alternative to end-to-end LLM approaches, providing a scalable AI solution tailored for enterprise applications.

Abstract

Enterprise workloads are dominated by deterministic, structured, and knowledge-dependent tasks operating under strict cost, latency, and reliability constraints. While these are often addressed through large language model (LLM) deployment or distillation into smaller models, we argue this is inefficient, unreliable, and misaligned with enterprise task structures. Instead, AI systems should treat language models as interfaces rather than monolithic engines, externalizing knowledge and computation into dedicated components for greater reliability, scalability, and transparency. Our theoretical evidences show that finite-capacity models cannot fully capture the breadth of knowledge required for enterprise tasks, creating inherent limits to efficiency and interpretability. Building on this, we take the position that language models should primarily be used for structured extraction in deterministic enterprise workflows, while computation and storage are delegated to knowledge bases and symbolic procedures. We formally demonstrate that such modular architectures are more reliable and maintainable than monolithic frameworks, offering a sustainable foundation for enterprise tasks.

Problem

• enterprise tasks
• large language models
• deterministic workflows
• knowledge dependency
• reliability constraints

Innovation

• modular architecture
• large language models
• enterprise AI
• knowledge externalization
• structured extraction