From Images to Decisions: Assistive Computer Vision for Non-Metallic Content Estimation in Scrap Metal

2026-02-05

AI Summary

This study addresses the subjectivity and safety hazards associated with manual visual assessment of non-metallic impurities in scrap steel, which adversely affect steelmaking energy efficiency and emissions. The authors propose the first computer vision system integrating multiple instance learning (MIL) and multi-task learning (MTL) to model impurity evaluation as a regression task based on unloading images. The system simultaneously predicts impurity percentage (MAE 0.27, R² 0.83) and scrap type (F1 0.79). Innovatively, it incorporates a closed-loop active learning mechanism and temporal image segmentation, and deploys versioned inference services for seamless integration into industrial acceptance and melting planning workflows. This approach significantly reduces human bias while enhancing automation and operational safety.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Web data integration and cleaning
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Scrap quality directly affects energy use, emissions, and safety in steelmaking. Today, the share of non-metallic inclusions (contamination) is judged visually by inspectors - an approach that is subjective and hazardous due to dust and moving machinery. We present an assistive computer vision pipeline that estimates contamination (per percent) from images captured during railcar unloading and also classifies scrap type. The method formulates contamination assessment as a regression task at the railcar level and leverages sequential data through multi-instance learning (MIL) and multi-task learning (MTL). Best results include MAE 0.27 and R2 0.83 by MIL; and an MTL setup reaches MAE 0.36 with F1 0.79 for scrap class. Also we present the system in near real time within the acceptance workflow: magnet/railcar detection segments temporal layers, a versioned inference service produces railcar-level estimates with confidence scores, and results are reviewed by operators with structured overrides; corrections and uncertain cases feed an active-learning loop for continual improvement. The pipeline reduces subjective variability, improves human safety, and enables integration into acceptance and melt-planning workflows.

Problem

• non-metallic contamination
• scrap metal
• visual inspection
• subjectivity
• safety hazard

Innovation

• multi-instance learning
• multi-task learning
• computer vision
• active learning
• scrap metal contamination