Cross-Core Inference Offload as an Operating-System Service on Dual-Core Microcontrollers

2026-07-14

AI Summary

This work addresses the challenge of executing AI inference on resource-constrained auxiliary cores that lack the capability to run AI workloads directly. The authors propose an asymmetric dual-core microcontroller architecture enabling cross-core inference offloading: the AI runtime is deployed on a fully featured main core and exposed to the auxiliary core as a native operating system service for remote inference invocation. The key innovation lies in implementing cross-core inference as an OS-native service, supporting zero-copy tensor transfer, priority-based scheduling, error and timeout propagation, and autonomous recovery after auxiliary core resets. Built upon Zephyr, the resulting SynapticOS employs lock-free single-producer/single-consumer ring buffers, memory barrier synchronization, and MPU-based isolation to ensure efficient and secure inter-core communication. Evaluated on the FRDM-MCXN947 platform, the system achieves a 15 μs round-trip latency (within a 50 μs budget), incurs zero errors across 1,913 dual-model stress tests, occupies only 98.9 KB of flash, and passes all 108 functional tests.

Abstract

Dual-core MCUs are asymmetric: on NXP's MCXN947, the second Cortex-M33 has no FPU, DSP extension, TrustZone, or MPU. We treat the asymmetry as a design input in the Phase 3 dual-core architecture of SynapticOS, an open-source Zephyr-based runtime: the AI runtime (models, NPU/DSP, scheduler) lives on the capable core, and the application core reaches inference only via a message-based OS service -- a remote system call. The transport is a pair of lock-free single-producer/single-consumer rings in shared SRAM: one writer per index, free-running 32-bit counters, ordering by data-memory barriers alone (the platform has no cross-core atomics). Because ring state is shared, a rebooting application core rejoins unaided. Requests carry priority classes, errors and timeouts propagate to the caller, and tensors stage zero-copy in a shared slot -- a 27 KB frame cannot exist twice in 64 KB of RAM. Measured on the FRDM-MCXN947 (both cores 150 MHz): the application core boots in 1,514 us and completes the handshake in 2,554 us, bit-identical over 11 boots; round trips are 15 us typical / 81 us worst-case (50 us budget); pushes cost 25 cycles; a 1,913-serve two-model soak had zero errors (stub-NPU latencies bracket transport, not silicon). An MPU region on the runtime core guards the application core's RAM (fault-injection verified); protection is one-directional -- the application core has no MPU, and ARMv8-M cannot block privileged reads. Two hardware-revealed defects are reported: releasing the second core into erased flash wedges the whole chip and its debug port (now prevented by a ROM-API blank check), and a Zephyr flash-driver Kconfig silently disarmed the devicetree MPU guard (now programmed at runtime). Firmware is 98.9 KB flash (runtime core) and 32.4 KB (application core, 42.6 of 64 KB RAM); 108 tests in 13 suites pass 100%. Apache 2.0: https://github.com/Dimitrios-Kafetzis/SynapticOS

Problem

• inference offload
• asymmetric dual-core
• microcontroller
• cross-core communication
• resource-constrained systems

Innovation

• cross-core inference offload
• lock-free ring buffer
• zero-copy tensor staging
• asymmetric dual-core MCU
• OS-level remote syscall