COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation

2024-12-23

AI Summary

In few-shot adaptation, retrieval-augmented methods relying solely on nearest neighbors often introduce redundancy and suffer from low diversity. Method: This paper proposes a composite retrieval framework that jointly optimizes similarity and diversity. Its core innovation is the first introduction of Compositional Mutual Information (CMI) theory into retrieval augmentation, enabling a CMI-driven joint metric for simultaneous diversity-similarity optimization over candidate samples. The method operates on the LAION-2B image-text pool and employs a lightweight CMI optimization algorithm, facilitating plug-and-play integration into mainstream few-shot learning pipelines. Contribution/Results: Experiments demonstrate significant performance gains over nearest-neighbor baselines on few-shot image classification tasks, with negligible computational overhead and consistent downstream model improvement.

Technology Category

• Computer Vision: Image and Video Retrieval
• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Retrieval augmentation, the practice of retrieving additional data from large auxiliary pools, has emerged as an effective technique for enhancing model performance in the low-data regime. Prior approaches have employed only nearest-neighbor based strategies for data selection, which retrieve auxiliary samples with high similarity to instances in the target task. However, these approaches are prone to selecting highly redundant samples, since they fail to incorporate any notion of diversity. In our work, we first demonstrate that data selection strategies used in prior retrieval-augmented few-shot adaptation settings can be generalized using a class of functions known as Combinatorial Mutual Information (CMI) measures. We then propose COBRA (COmBinatorial Retrieval Augmentation), which employs an alternative CMI measure that considers both diversity and similarity to a target dataset. COBRA consistently outperforms previous retrieval approaches across image classification tasks and few-shot learning techniques when used to retrieve samples from LAION-2B. COBRA introduces negligible computational overhead to the cost of retrieval while providing significant gains in downstream model performance.

Problem

• Improves few-shot learning via diverse data retrieval
• Addresses redundancy in nearest-neighbor retrieval methods
• Enhances model performance with combinatorial mutual information

Innovation

• Uses Combinatorial Mutual Information measures
• Balances diversity and similarity in retrieval
• Minimal computational overhead with performance gains