From Artefact to Insight: Efficient Low-Rank Adaptation of BrushNet for Scanning Probe Microscopy Image Restoration

2026-03-16

AI Summary

Scanning probe microscopy (SPM) images are frequently degraded by structured artifacts such as line scan loss, gain noise, tip convolution, and phase jumps, which severely compromise nanoscale analysis accuracy. This work introduces generative image inpainting to the SPM domain for the first time, proposing an efficient diffusion-based restoration framework. By fine-tuning a pretrained BrushNet model using Low-Rank Adaptation (LoRA), the method updates only 0.2% of the parameters while achieving a 6.61 dB improvement in PSNR and a 50% reduction in LPIPS on the newly curated SPM InpBench benchmark—performance comparable to full-parameter fine-tuning but requiring only a single GPU for training. The approach effectively suppresses hallucinatory artifacts induced by natural image priors and demonstrates strong cross-channel generalization and high scalability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Scanning Probe Microscopy or SPM offers nanoscale resolution but is frequently marred by structured artefacts such as line scan dropout, gain induced noise, tip convolution, and phase hops. While most available methods treat SPM artefact removal as isolated denoising or interpolation tasks, the generative inpainting perspective remains largely unexplored. In this work, we introduce a diffusion based inpainting framework tailored to scientific grayscale imagery. By fine tuning less than 0.2 percent of BrushNet weights with rank constrained low rank adaptation (LoRA), we adapt a pretrained diffusion model using only 7390 artefact, clean pairs distilled from 739 experimental scans. On our forthcoming public SPM InpBench benchmark, the LoRA enhanced model lifts the Peak Signal to Noise Ratio or PSNR by 6.61 dB and halves the Learned Perceptual Image Patch Similarity or LPIPS relative to zero-shot inference, while matching or slightly surpassing the accuracy of full retraining, trainable on a single GPU instead of four high-memory cards. The approach generalizes across various SPM image channels including height, amplitude and phase, faithfully restores subtle structural details, and suppresses hallucination artefacts inherited from natural image priors. This lightweight framework enables efficient, scalable recovery of irreplaceable SPM images and paves the way for a broader diffusion model adoption in nanoscopic imaging analysis.

Problem

• Scanning Probe Microscopy
• image artefacts
• structured artefacts
• image restoration
• generative inpainting

Innovation

• Low-Rank Adaptation
• Diffusion Inpainting
• Scanning Probe Microscopy
• BrushNet
• Scientific Image Restoration