NeurIDA: Dynamic Modeling for Effective In-Database Analytics

2025-12-09

AI Summary

Traditional machine learning models are static and task-specific, rendering them ill-suited for the dynamic and heterogeneous analytical demands of relational databases—leading to high development overhead and poor deployability. This paper proposes an in-database dynamic AI analytics paradigm: it interprets user intent via natural language interaction, automatically generates task configurations, and dynamically composes pretrained components from a shared foundation model pool to adapt to diverse analytical tasks; it further integrates an LLM-based agent to generate interpretable analytical reports. Key innovations include a composable foundation model architecture, a task-agnostic model reuse mechanism, and a dynamic orchestration framework. Experiments across five real-world datasets and ten analytical tasks demonstrate that our approach achieves up to a 12% improvement in AUC-ROC and a 25% relative reduction in MAE over static modeling paradigms, significantly lowering the barrier to AI-powered database analytics.

Technology Category

• Machine Learning: Statistical Relational/Logic Learning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Data Mining & Knowledge Management: Intelligent Query Processing

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Relational Database Management Systems (RDBMS) manage complex, interrelated data and support a broad spectrum of analytical tasks. With the growing demand for predictive analytics, the deep integration of machine learning (ML) into RDBMS has become critical. However, a fundamental challenge hinders this evolution: conventional ML models are static and task-specific, whereas RDBMS environments are dynamic and must support diverse analytical queries. Each analytical task entails constructing a bespoke pipeline from scratch, which incurs significant development overhead and hence limits wide adoption of ML in analytics. We present NeurIDA, an autonomous end-to-end system for in-database analytics that dynamically "tweaks" the best available base model to better serve a given analytical task. In particular, we propose a novel paradigm of dynamic in-database modeling to pre-train a composable base model architecture over the relational data. Upon receiving a task, NeurIDA formulates the task and data profile to dynamically select and configure relevant components from the pool of base models and shared model components for prediction. For friendly user experience, NeurIDA supports natural language queries; it interprets user intent to construct structured task profiles, and generates analytical reports with dedicated LLM agents. By design, NeurIDA enables ease-of-use and yet effective and efficient in-database AI analytics. Extensive experiment study shows that NeurIDA consistently delivers up to 12% improve- ment in AUC-ROC and 25% relative reduction in MAE across ten tasks on five real-world datasets. The source code is available at https://github.com/Zrealshadow/NeurIDA

Problem

• Dynamic ML model adaptation for diverse RDBMS analytical tasks
• Reducing development overhead in integrating machine learning with databases
• Enabling natural language queries for user-friendly in-database analytics

Innovation

• Dynamic in-database modeling with composable base model architecture
• Automatically selects and configures model components based on task profiles
• Supports natural language queries using LLM agents for user-friendly analytics