TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

2026-06-24

AI Summary

This work addresses the high communication overhead in federated learning, the inability of conventional split learning to replicate centralized mini-batch gradient dynamics, and its associated privacy risks by proposing TL++, a dual-mode traversing learning framework. TL++ constructs virtual batches across nodes to faithfully reproduce centralized training dynamics. In its base mode, it exchanges only activations and gradients at the cut layer; in its secure mode, it further integrates secret sharing to provide activation-level privacy guarantees. TL++ achieves, for the first time in split learning, accuracy nearly matching that of centralized training—reaching 91.41% (base) and 90.93% (secure) on CIFAR-10—outperforming the strongest baseline by over 12 percentage points while reducing communication overhead by 13.1×, and demonstrates strong performance on the PubMedQA task as well.

Abstract

Distributed intelligent systems increasingly need to train across data silos without centralizing raw data. Federated learning keeps data local but can suffer under heterogeneous partitions and requires repeated full-model exchange. Split learning reduces communication through cut-layer activations, but standard protocols generally do not recover centralized mini-batch gradient behavior and may expose activations and gradients in plaintext. We present TL++, a two-mode traversal-learning framework that constructs virtual batches across nodes to recover centralized mini-batch gradient behavior under explicit synchronization assumptions. Base mode exchanges cut-layer activations and gradients rather than full models. Secure mode secret-shares each cut-layer activation and gradient between an orchestrator and a non-colluding helper, preventing either server from observing plaintext cut-layer tensors. This protection is limited to a semi-honest two-server setting; labels and loss-related outputs remain visible to the orchestrator. In the lightweight secure path evaluated here, exactness requires a linear or affine server path, while nonlinear operations require nonlinear MPC or approximation. We formalize TL++, analyze communication and computation costs, and evaluate it against federated and split-learning baselines on CIFAR-10 and BioGPT/PubMedQA using full fine-tuning and LoRA. On CIFAR-10, TL++ base cut 1 and exact secure cut 3 achieve accuracies of 91.41% (SD 0.19) and 90.93% (SD 0.17), respectively, exceeding the strongest measured non-TL++ baseline by more than 12 percentage points. TL++ base cut 1 also reduces per-step communication by 13.1-fold relative to full-model synchronization. PubMedQA results similarly favor TL++. Overall, TL++ approaches centralized-training performance while reducing communication and providing activation-level secret sharing.

Problem

• federated learning
• split learning
• privacy preservation
• distributed intelligent systems
• data silos

Innovation

• Traversal Learning
• Privacy Preservation
• Secret Sharing
• Distributed Intelligence
• Communication Efficiency