AURORA: Adaptive Unified Representation for Robust Ultrasound Analysis

2026-03-19

AI Summary

This work addresses the limited generalization of deep learning models on ultrasound images, which arises from variations in imaging devices, operators, and anatomical targets. To this end, the authors propose a unified multi-task framework built upon the Qwen3-VL vision encoder. The approach integrates segmentation, detection, classification, and keypoint regression within a single end-to-end architecture by leveraging intermediate token feature projection and a lightweight multi-scale feature pyramid fusion module. Task-aware sampling and a selective loss balancing strategy are further introduced to harmonize heterogeneous supervisory signals. Experimental results demonstrate substantial improvements in cross-hospital and cross-task generalization, with validation accuracy increasing from 67% to 85% and achieving an average score of 81.84% on the official test set.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Ultrasound images vary widely across scanners, operators, and anatomical targets, which often causes models trained in one setting to generalize poorly to new hospitals and clinical conditions. The Foundation Model Challenge for Ultrasound Image Analysis (FMC-UIA) reflects this difficulty by requiring a single model to handle multiple tasks, including segmentation, detection, classification, and landmark regression across diverse organs and datasets. We propose a unified multi-task framework based on a transformer visual encoder from the Qwen3-VL family. Intermediate token features are projected into spatial feature maps and fused using a lightweight multi-scale feature pyramid, enabling both pixel-level predictions and global reasoning within a shared representation. Each task is handled by a small task-specific prediction head, while training uses task-aware sampling and selective loss balancing to manage heterogeneous supervision and reduce task imbalance. Our method is designed to be simple to optimize and adaptable across a wide range of ultrasound analysis tasks. The performance improved from 67% to 85% on the validation set and achieved an average score of 81.84% on the official test set across all tasks. The code is publicly available at: https://github.com/saitejalekkala33/FMCUIA-ISBI.git

Problem

• ultrasound image analysis
• domain generalization
• multi-task learning
• foundation model
• clinical variability

Innovation

• unified multi-task learning
• transformer visual encoder
• feature pyramid fusion
• task-aware sampling
• ultrasound image analysis