AI RESEARCH
Profiling-Driven Adaptive Distributed Transformer Inference on Embedded Edge Deployment
arXiv CS.AI
•
ArXi:2605.25682v1 Announce Type: cross Distributing Transformer inference across embedded edge devices can alleviate individual memory and compute constraints, yet practical benefits on real hardware remain unclear: prior work relies largely on simulations that overlook hardware-specific communication overheads. We present a hardware prototype study on NVIDIA Jetson Orin Nano devices connected over WiFi. Our key finding is that the dominant bottleneck is not just network bandwidth but also the CPU-GPU staging during communication.