Active Learning for Efficient Annotation of Surgical Videos with Weak Supervision

2026-07-14

AI Summary

This work addresses the high cost and expert dependency of fine-grained spatiotemporal annotation in laparoscopic videos, which hinders the development of surgical instrument segmentation models. The authors propose a human-in-the-loop weakly supervised learning framework that, for the first time, integrates temporally consistent class activation maps generated by foundation models with active learning. By iteratively refining dual-objective losses—comprising video-level classification and pixel-level segmentation losses—using only video-level weak labels and expert-corrected image-level masks, the method progressively generates high-quality pseudo-masks. This approach substantially reduces annotation burden, achieving comparable performance with 50% less manual labeling effort, and enables efficient scaling to large-scale clinical datasets.

Abstract

Precise spatial-temporal annotation of laparoscopic videos is time-consuming and requires expert knowledge. We propose a human-in-the-loop knowledge acquisition framework that combines active learning with dual-loss optimization to significantly reduce the annotation effort needed for automatic localization and segmentation of objects in the surgical field. Our method employs a foundation model to generate temporally consistent class activation maps (CAMs) from video using two complementary training objectives: a weak supervision loss on video-level tool presence labels for weakly annotated data, and an image-level mask loss on human-corrected annotations obtained through active learning. Rather than requiring dense pixel-level annotation upfront, our pipeline iteratively proposes pseudo-masks that guide the expert annotator to refine the knowledge previously captured by the model. We demonstrate that our framework reduces the effort of surgical video annotation by 50% by the end of training in comparison to fully manual annotation. Through eliminating the need for large, fully annotated datasets from the start, this framework enables scalability to the development of surgical tool segmentation models. This iterative human-in-the-loop refinement supports efficient knowledge acquisition with minimal expert input, providing a practical and deployable strategy for expanding tool segmentation to larger, more diverse datasets and real-world clinical settings.

Problem

• surgical video annotation
• active learning
• weak supervision
• tool segmentation
• human-in-the-loop

Innovation

• active learning
• weak supervision
• dual-loss optimization
• surgical video annotation
• human-in-the-loop