Kernel-FFI: Transparent Foreign Function Interfaces for Interactive Notebooks

2025-07-30

AI Summary

Existing foreign function interface (FFI) solutions struggle to support dynamic workflows in interactive notebooks (e.g., Jupyter): they require manual configuration, extensive boilerplate code, and lack support for recursive calls and cross-language object-oriented programming (OOP). This paper proposes TransFFI—a transparent, language-agnostic FFI framework that automatically enables cross-language function invocation and object manipulation via source-level parsing and rewriting, eliminating hand-written bindings. Its key innovations include a side-channel communication mechanism that avoids kernel blocking, enabling recursive and asynchronous cross-language calls; and unified cross-language object serialization with lifetime-aware resource management, ensuring reference preservation and automatic memory reclamation. Evaluated across multi-language notebook environments, TransFFI significantly improves developer productivity and interactive flexibility while maintaining type safety and runtime efficiency.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Foreign Function Interfaces (FFIs) are essential for enabling interoperability between programming languages, yet existing FFI solutions are ill-suited for the dynamic, interactive workflows prevalent in modern notebook environments such as Jupyter. Current approaches require extensive manual configuration, introduce significant boilerplate, and often lack support for recursive calls and object-oriented programming (OOP) constructs-features critical for productive, multi-language development. We present Kernel-FFI, a transparent, language-agnostic framework that enables seamless cross-language function calls and object manipulation within interactive notebooks. Kernel-FFI employs source-level transformation to automatically rewrite cross-language invocations, eliminating the need for manual bindings or boilerplate. Kernel-FFI provides robust support for OOP by enabling foreign object referencing and automatic resource management across language boundaries. Furthermore, to address the blocking nature of Jupyter kernels and support recursive and asynchronous foreign calls, we introduce a novel side-channel communication mechanism. Our tool will be open-sourced and available at https://codepod.io/docs/kernel-ffi

Problem

• Enables seamless cross-language calls in notebooks
• Eliminates manual bindings and boilerplate code
• Supports OOP and recursive calls across languages

Innovation

• Transparent language-agnostic framework for notebooks
• Source-level transformation eliminates manual bindings
• Side-channel communication supports recursive asynchronous calls