HugSelect: An Explainable Multi-Criteria Decision-Support Framework for foundation-model selection

2026-08-08

AI Summary

This work addresses the lack of systematic, auditable decision support in foundational model selection, a process often driven by popularity rather than functional capabilities, operational constraints, or community-based quality assessments. To remedy this, we propose HugSelect—the first interpretable multi-criteria decision framework that formalizes foundational model selection as a traceable software component selection task. HugSelect integrates model metadata, functional attributes, and community-perceived quality into a unified knowledge base and a decomposable weighted scoring system. Evaluated on 71,274 models, our approach achieves an F1 score of 0.801 in functional feature extraction and 0.84 accuracy in quality mapping. It attains model-level Coverage@10 of 0.61 and family-level Coverage@10 of 0.91, matching the recommendation performance of leading commercial systems while offering transparent, interpretable results that elicit positive user feedback.

Abstract

Foundation models are increasingly reused as software components, making model selection a critical software-engineering decision. Current model hubs primarily support discovery through popularity metrics, often neglecting functional capabilities, operational constraints, and community-perceived quality. We argue that foundation-model selection should be treated as an explicit, auditable software-component selection task rather than as keyword search, popularity ranking, or opaque conversational advice. This paper proposes HugSelect, an explainable decision-support framework for foundation-model selection. HugSelect builds a knowledge base of 71,274 models by combining repository metadata, extracted functional capabilities, and perceived quality attributes derived from community discussions into a unified pipeline. It ranks candidate models using a weighted additive model that exposes criterion-level score decompositions. We evaluated HugSelect through pipeline validation, comparative case studies against four commercial LLM-based recommendation systems (44 scenarios), fine-grained ablation, and an exploratory user study (n = 10). Extraction pipelines achieved an F1 score of 0.801 for functional features and an accuracy of 0.84 for quality-attribute mapping. HugSelect achieved a model-level Coverage@10 of 0.61 and family-level Coverage@10 of 0.91, showing recommendation quality comparable to that of the evaluated commercial systems, with no significant overall differences in ranking quality, while providing stable, traceable, and inspectable reasoning. Ablation confirmed that functional features were the main driver of retrieval accuracy, and preliminary user feedback suggests that the framework is useful and intuitive.

Problem

• foundation-model selection
• multi-criteria decision support
• explainable AI
• software-component selection
• model hubs

Innovation

• explainable AI
• foundation model selection
• multi-criteria decision support
• functional capability extraction
• community-perceived quality