Explainable and Class-Revealing Signal Feature Extraction via Scattering Transform and Constrained Zeroth-Order Optimization

2025-02-08

AI Summary

This work addresses the interpretability bottleneck of joint scattering transform–multiclass logistic regression models, wherein nonlinear scattering features resist semantic interpretation. We propose an inversion method based on constrained zeroth-order optimization to reconstruct class-specific prototype input signals that yield target scattering coefficients. Our key contribution is the first integration of sparsity (Lasso) and temporal smoothness (total variation) regularization into Natural Evolution Strategies (NES)-type zeroth-order optimization, enabling stable inversion and semantic decoupling of scattering coefficients for class attribution. The method is gradient-free and applicable to non-differentiable models. Evaluated on multiple synthetic time-series classification tasks, it successfully visualizes physically meaningful, discriminative signal patterns—thereby enhancing decision transparency and improving the reliability of feature attribution.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Computer Vision: Interpretability, Explainability, and Transparency
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

We propose a new method to extract discriminant and explainable features from a particular machine learning model, i.e., a combination of the scattering transform and the multiclass logistic regression. Although this model is well-known for its ability to learn various signal classes with high classification rate, it remains elusive to understand why it can generate such successful classification, mainly due to the nonlinearity of the scattering transform. In order to uncover the meaning of the scattering transform coefficients selected by the multiclass logistic regression (with the Lasso penalty), we adopt zeroth-order optimization algorithms to search an input pattern that maximizes the class probability of a class of interest given the learned model. In order to do so, it turns out that imposing sparsity and smoothness of input patterns is important. We demonstrate the effectiveness of our proposed method using a couple of synthetic time-series classification problems.

Problem

• Extract explainable features from machine learning model
• Understand scattering transform coefficients via optimization
• Enhance classification with sparsity and smoothness constraints

Innovation

• Scattering transform feature extraction
• Constrained zeroth-order optimization
• Sparse smooth input patterns