Institution profile

University of Zurich

Academic institutioneurope · ch
Official website
Research library732linked papers
Opportunities0open roles
Selected work

Representative Papers

Developing Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomography

Mar 26, 2024

Current 3D medical imaging AI is hindered by the scarcity of large-scale, paired multimodal datasets, impeding cross-modal alignment and natural language interaction. To address this, we introduce CT-RATE—the first large-scale, paired 3D chest CT–radiology report dataset (25,692 cases)—and propose CT-CLIP, a contrastive learning framework, and CT-CHAT, a vision-language dialogue model. Our contributions include: (1) the first large-scale, fine-grained alignment between 3D CT volumes and free-text radiology reports; (2) CT-CLIP—a task-agnostic foundational model integrating 3D convolutional networks with Vision Transformers, requiring no downstream fine-tuning; and (3) CT-CHAT—the first open-source, 3D CT–specific conversational model, trained via report-driven QA generation and LLM-CT co-fine-tuning for end-to-end diagnostic interaction. Experiments demonstrate that our unsupervised multi-abnormality detection outperforms fully supervised SOTA methods; cross-modal retrieval enables bidirectional image–text queries; and CT-CHAT, fine-tuned on 2.7M medical QA pairs, surpasses existing multimodal medical assistants.

24 citations3 influentialRead paper

Benchmarking the CoW with the TopCoW Challenge: Topology-Aware Anatomical Segmentation of the Circle of Willis for CTA and MRA

Dec 29, 2023arXiv.org

The Circle of Willis (CoW) suffers from a scarcity of high-quality voxel-level annotations in CTA/MRA imaging, reliance on labor-intensive expert manual segmentation, and poor guarantee of topological consistency. Method: We introduce the first publicly available voxel-level multi-class CoW dataset—comprising 13 vascular structures with paired MRA/CTA volumes—and propose a topology-aware segmentation framework: (i) a novel VR-assisted annotation paradigm ensuring anatomical plausibility; (ii) a multimodal registration and topology-constrained segmentation network; and (iii) topology-sensitive metrics including branch F1 and topo-Dice. Contribution/Results: This benchmark has attracted >140 teams across four continents. State-of-the-art models achieve ≈90% Dice on most arterial branches, while exposing persistent topological matching bottlenecks—particularly for communicating arteries and anatomical variants.

24 citations2 influentialRead paper

Information theory for data-driven model reduction in physics and biology

Dec 11, 2023bioRxiv

This study addresses the fundamental challenge in multiscale dynamical systems modeling: *how to automatically identify a small set of critical slow variables to construct interpretable and predictive low-dimensional models*. To this end, we propose a data-driven dimensionality reduction framework grounded in the information bottleneck principle. Methodologically, we establish, for the first time, an analytical connection between slow variables and eigenfunctions of the Koopman (or transfer) operator; introduce an optimal truncation criterion based on information compression rate; and integrate variational inference with autoencoding neural networks to build an interpretable deep learning architecture capable of discovering emergent order parameters. Applied to satellite atmospheric flow videos, our method successfully extracts dominant slow variables; applied to experimental videos of cyanobacterial microcolonies, it uncovers a novel synchronization order parameter. The framework thus achieves a unified balance between model interpretability and predictive accuracy.

5 citationsRead paper

On Nonparanormal Likelihoods

Aug 30, 2024

To address statistical efficiency loss and standard error bias arising from the conventional two-stage approach—first estimating marginal distributions nonparametrically/semiparametrically, then fitting a Gaussian copula—in modeling multivariate non-normal data, this paper proposes an integrated likelihood framework that jointly estimates marginal distributions and Gaussian copula parameters. Key contributions include: (i) the first formal definition of four classes of nonparametric normal log-likelihood functions; (ii) identification and exploitation of the biconvex structure of the objective function, enabling a convex approximation optimization strategy; and (iii) derivation of exact score functions via the Genz algorithm, facilitating first-order optimization. The method substantially enhances robustness of transformation-based discriminant analysis for limit-of-detection biomarker data and improves asymptotic efficiency and standard error accuracy in semiparametric polychoric correlation estimation.

3 citations1 influentialRead paper

Learning Quadrotor Control From Visual Features Using Differentiable Simulation

Oct 21, 2024arXiv.org

Reinforcement learning (RL) for vision-driven quadrotor control suffers from low sample efficiency and heavy reliance on precise state feedback, resulting in slow training. Method: This paper proposes an end-to-end vision-based closed-loop control framework built upon differentiable simulation. It innovatively integrates a lightweight gradient surrogate model with joint state representation and policy learning, enabling rapid attitude recovery using only image features—without access to ground-truth state feedback. Differentiable physics simulation, visual feature encoding, and gradient-guided optimization collectively accelerate policy convergence and enhance cross-scenario generalization. Results: Experiments demonstrate that the method achieves pure vision-based attitude control within minutes of training—improving sample efficiency by over one order of magnitude compared to standard model-free RL baselines—thereby establishing a new paradigm for low-sample-cost vision-guided UAV control.

2 citationsRead paper
Recent publications

Latest Papers