Muon-C: Operator-Aligned Muon for Convolutional Kernels

2026-09-08

AI Summary

本文提出Muon-C，一种针对卷积核优化的方法，通过频率通道转移矩阵表示动量并独立极化，使用关键傅里叶网格精确更新。实验表明其在CIFAR-10等任务上优于传统方法。

Abstract

Muon replaces matrix momentum with an approximately orthogonal polar direction, but its geometry depends on the matrix representation. For convolution, standard unfolding describes a local patch map rather than the convolution operator. We introduce Muon-C, an operator-aligned optimizer that represents kernel momentum as frequency-wise channel-transfer matrices, polarizes these blocks independently, and uses a critical Fourier grid to return updates exactly to the original finite kernel support. We show that the new geometry arises from combining the block partition and Fourier coordinates. The exact-polar direction is a linear minimization oracle under the critically sampled convolution norm. Its worst-case guarantee relative to the continuous convolution-operator norm is never weaker than unfolding and is strictly stronger for $3\times3$ kernels. On CIFAR-10 flow matching with matched applied-update RMS, Muon-C reaches 9.87 FID at 40k iterations, compared with 22.26 for unfolded Muon and 51.31 for Adam. It reaches their final quality using $0.62\times$ and $0.64\times$ their model FLOPs, respectively. Under equal tuning budgets, Muon-C achieves 3.42 FID. Gains persist across data scales and transfer to classification across convolutional architectures.

Problem

• convolution
• optimizer
• momentum
• Fourier grid
• unfolding

Innovation

• Operator-Aligned
• Convolutional Kernels
• Fourier Grid
• Polar Direction
• Channel-Transfer Matrices