🤖 AI Summary
This work addresses the high computational cost of training and inference in remote sensing models by introducing an efficient architecture design, optimized training strategies, and tailored Sentinel-2 data processing techniques to significantly enhance the computational efficiency of the OlmoEarth model. Without compromising model performance, the proposed approach reduces GPU-hours required for training the Base model by 1.7× and decreases inference MACs (Multiply-Accumulate operations) on Sentinel-2 tasks by 2.9×, thereby substantially lowering the computational burden of remote sensing applications.
📝 Abstract
We present a set of improvements to the OlmoEarth family. These improvements allow us to cut compute costs during training ($1.7 \times$ reduction in GPU hours required to train our Base models) and inference ($2.9\times$ reductions in MACs on Sentinel-2 tasks), while maintaining the models' overall performance. All training code is available at github.com/allenai/olmoearth_pretrain.