Multimodal Survival Analysis with Locally Deployable Large Language Models

2026-03-23

AI Summary

This work addresses the challenge of multimodal survival analysis in healthcare settings constrained by limited computational resources and stringent privacy requirements. The authors propose a lightweight, locally deployable large language model (LLM)-based framework that fuses clinical text, tabular covariates, and genomic data through a teacher–student distillation approach. This method jointly calibrates survival probabilities and generates evidence-driven prognostic explanations without relying on cloud infrastructure. To the best of the authors’ knowledge, this is the first study to integrate a locally executable LLM into survival analysis, effectively balancing privacy preservation, predictive accuracy, and interpretability. Experiments on The Cancer Genome Atlas (TCGA) cohorts demonstrate that the proposed approach significantly outperforms existing baselines.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We study multimodal survival analysis integrating clinical text, tabular covariates, and genomic profiles using locally deployable large language models (LLMs). As many institutions face tight computational and privacy constraints, this setting motivates the use of lightweight, on-premises models. Our approach jointly estimates calibrated survival probabilities and generates concise, evidence-grounded prognosis text via teacher-student distillation and principled multimodal fusion. On a TCGA cohort, it outperforms standard baselines, avoids reliance on cloud services and associated privacy concerns, and reduces the risk of hallucinated or miscalibrated estimates that can be observed in base LLMs.

Problem

• multimodal survival analysis
• large language models
• privacy constraints
• clinical text
• genomic profiles

Innovation

• multimodal survival analysis
• locally deployable LLMs
• teacher-student distillation
• privacy-preserving AI
• evidence-grounded prognosis