Physical Foundation Models: Fixed hardware implementations of large-scale neural networks

2026-04-30

AI Summary

Traditional AI hardware is constrained by low energy efficiency, slow processing speeds, and insufficient parameter density, hindering the deployment of trillion-parameter foundation models in both data centers and edge devices. This work introduces, for the first time, the concept of a “Physical Foundation Model” (PFM), which transcends conventional digital paradigms by directly embedding large-scale neural networks into physical media—such as three-dimensional nanostructured glass—and leveraging their intrinsic dynamics to perform inference. By integrating optical nanostructures with nanoelectronic platforms and combining read-only weights with in-situ physical computation, PFM theoretically enables order-of-magnitude improvements in energy efficiency, computational speed, and parameter density, thereby offering a novel pathway toward deploying hundred-trillion-parameter models and empowering edge intelligence.

Abstract

Foundation models are deep neural networks (such as GPT-5, Gemini~3, and Opus~4) trained on large datasets that can perform diverse downstream tasks -- text and code generation, question answering, summarization, image classification, and so on. The philosophy of foundation models is to put effort into a single, large (${\sim}10^{12}$-parameter) general-purpose model that can be adapted to many downstream tasks with no or minimal additional training. We argue that the rise of foundation models presents an opportunity for hardware engineers: in contrast to when different models were used for different tasks, it now makes sense to build special-purpose, fixed hardware implementations of neural networks, manufactured and released at the roughly 1-year cadence of major new foundation-model versions. Beyond conventional digital-electronic inference hardware with read-only weight memory, we advocate a more radical re-thinking: hardware in which the neural network is realized directly at the level of the physical design and operates via the hardware's natural physical dynamics -- \textit{Physical Foundation Models} (PFMs). PFMs could enable orders-of-magnitude advantages in energy efficiency, speed, and parameter density. For ${\sim}10^{12}$-parameter models, this would both reduce the high energy burden of AI in datacenters and enable AI in edge devices that today are power-constrained to far smaller models. PFMs could also enable inference hardware for models much larger than current ones: $10^{15}$- or even $10^{18}$-parameter PFMs seem plausible by some measures. We present back-of-the-envelope calculations illustrating PFM scaling using an optical example -- a 3D nanostructured glass medium -- and discuss prospects in nanoelectronics and other physical platforms. We conclude with the major research challenges that must be resolved for trillion-parameter PFMs and beyond to become reality.

Problem

• Foundation Models
• Physical Hardware
• Energy Efficiency
• Parameter Scaling
• Edge AI

Innovation

• Physical Foundation Models
• hardware acceleration
• neuromorphic computing
• optical computing
• energy-efficient AI