Institution profile

University of Tübingen

Academic institutioneurope · de
Official website
Research library637linked papers
Opportunities0open roles
Selected work

Representative Papers

Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metrics

Feb 17, 2025

Neural perceptual metrics like CLIP exhibit insufficient robustness against adversarial attacks, limiting their reliability in safety-critical zero-shot evaluation tasks. Method: We propose R-CLIP$_ extrm{F}$, an unsupervised adversarial fine-tuning framework that jointly optimizes CLIP’s feature space for robustness and performs self-supervised adaptation under adversarial perturbations—without requiring labeled data. It further introduces feature and text inversion mechanisms to enhance interpretability and enable visual concept visualization. Contribution/Results: R-CLIP$_ extrm{F}$ is the first method achieving both high robustness and high discriminability in zero-shot perceptual similarity modeling. Experiments demonstrate its superior performance over state-of-the-art metrics on zero-shot perceptual assessment, robust vision–language retrieval, and NSFW content detection. Crucially, it maintains high accuracy under adversarial perturbations while preserving original-image performance—bridging the robustness–accuracy trade-off without supervision.

3 citations1 influentialRead paper

Adding Internal Audio Sensing to Internal Vision Enables Human-Like In-Hand Fabric Recognition with Soft Robotic Fingertips

Sep 30, 2025IEEE-RAS International Conference on Humanoid Robots

Distinguishing the feel of smooth silk from coarse cotton is a trivial everyday task for humans. When exploring such fabrics, fingertip skin senses both spatio-temporal force patterns and texture-induced vibrations that are integrated to form a haptic representation of the explored material. It is challenging to reproduce this rich, dynamic perceptual capability in robots because tactile sensors typically cannot achieve both high spatial resolution and high temporal sampling rate. In this work, we present a system that can sense both types of haptic information, and we investigate how each type influences robotic tactile perception of fabrics. Our robotic hand's middle finger and thumb each feature a soft tactile sensor: one is the opensource Minsight sensor that uses an internal camera to measure fingertip deformation and force at 50 Hz, and the other is our new sensor Minsound that captures vibrations through an internal MEMS microphone with a bandwidth from 50 Hz to 15 kHz. Inspired by the movements humans make to evaluate fabrics, our robot actively encloses and rubs folded fabric samples between its two sensitive fingers. Our results test the influence of each sensing modality on overall classification performance, showing high utility for the audio-based sensor. Our transformer-based method achieves a maximum fabric classification accuracy of 97% on a dataset of 20 common fabrics. Incorporating an external microphone away from Minsound increases our method's robustness in loud ambient noise conditions. To show that this audio-visual tactile sensing approach generalizes beyond the training data, we learn general representations of fabric stretchiness, thickness, and roughness.

2 citationsRead paper

Neural timescales from a computational perspective

Sep 04, 2024arXiv.org

This study addresses the definition, measurement, mechanisms, and functional roles of neural timescales in neural computation and brain function. We propose a unified tripartite framework integrating data-driven quantification, biophysically grounded modeling, and functional validation—combining multimodal neural recordings, leaky integrate-and-fire (LIF) and adaptive exponential (AdEx) spiking network models, task-optimized recurrent neural networks (RNNs) and LSTMs, information-theoretic analyses, and causal inference methods. We systematically clarify theoretical distinctions among existing timescale estimation techniques and, for the first time, establish a causal link between slow membrane time constants and hierarchical computational capacity. Furthermore, we demonstrate that neural timescales serve as a necessary core variable mediating structure–dynamics–behavior mappings. These findings provide a methodological benchmark for standardized neural timescale characterization and lay a theoretical foundation for brain-inspired temporal processing architectures.

2 citationsRead paper

Four Facets of Forecast Felicity: Calibration, Predictiveness, Randomness and Regret

Jan 25, 2024arXiv.org

This work addresses two fundamental questions: “What constitutes a principled predictive evaluation metric?” and “How are existing metrics formally related?” We propose a unified evaluation framework grounded in game theory, introducing the first four-dimensional “predictive welfare” metric—comprising calibration, predictability, randomness, and regret—to holistically assess prediction quality. Theoretically, we rigorously prove the equivalence between calibration and regret, and establish a duality between predictive superiority and outcome randomness. Methodologically, we formalize the framework by integrating probabilistic calibration, regret analysis, and algorithmic randomness measures—specifically Martingale difference sequences. Our approach provides a more rigorous theoretical foundation for predictive evaluation, significantly enhancing both the interpretability and robustness assessment of trustworthy AI predictions.

2 citationsRead paper

Corruptions of Supervised Learning Problems: Typology and Mitigations

Jul 17, 2023

Existing data corruption studies are fragmented across specific scenarios, lacking a unified theoretical framework and systematic mitigation strategies. Method: We propose the first general corruption modeling framework based on Markov kernels, formalizing corruption as arbitrary modifications to the data distribution, hypothesis class, or loss function. We establish a provably complete taxonomy—distinguishing, for the first time, label corruption (affecting only the loss) from attribute or joint corruption (simultaneously affecting both the hypothesis class and the loss). Building on this, we introduce a generalized loss correction paradigm, deriving provably effective correction formulas for attribute and joint corruption under weaker assumptions than conventional approaches. Contribution/Results: Our framework unifies disparate corruption models and terminologies, providing a rigorous foundation for robustness analysis and algorithm design in supervised learning. It enables principled treatment of previously isolated corruption types and advances theoretical understanding of learning under distributional and structural perturbations.

2 citationsRead paper
Recent publications

Latest Papers