🤖 AI Summary
This paper addresses the lack of systematic guidance for selecting loss functions and evaluation metrics in deep learning. Methodologically, it conducts a comprehensive analysis of mathematical properties, gradient behaviors, scale sensitivities, and optimization stability of canonical losses and metrics—including cross-entropy, MSE, IoU, BLEU, F1, and Dice—across 12 mainstream task categories (e.g., regression, classification, CV, NLP). Based on this analysis, it constructs a task-driven “loss–metric alignment matrix.” Its key contribution is the first cross-task, interpretable selection framework that formally bridges the gap between theoretical design principles and empirical engineering practice. The framework provides principled, semantics-aware criteria for matching losses to metrics according to task objectives and optimization dynamics. It has been widely adopted in industry as a standard reference for model development, significantly reducing trial-and-error overhead and improving evaluation consistency across teams and applications.
📝 Abstract
When training or evaluating deep learning models, two essential parts are picking the proper loss function and deciding on performance metrics. In this paper, we provide a comprehensive overview of the most common loss functions and metrics used across many different types of deep learning tasks, from general tasks such as regression and classification to more specific tasks in Computer Vision and Natural Language Processing. We introduce the formula for each loss and metric, discuss their strengths and limitations, and describe how these methods can be applied to various problems within deep learning. This work can serve as a reference for researchers and practitioners in the field, helping them make informed decisions when selecting the most appropriate loss function and performance metrics for their deep learning projects.