AI RESEARCH
Cross-Platform Fused MoE Dispatch in Triton: Portable Expert Routing Without CUDA [R]
r/MachineLearning
•
New preprint. A Mixture-of-Experts inference kernel (TritonMoE) written entirely in OpenAI Triton, targeting portability across NVIDIA and AMD without vendor-specific code. Highlights: A fused gate+up GEMM computes both SwiGLU projections from shared tile loads, eliminating 35% of global memory traffic. 89-131% of Megablocks throughput at inference batch sizes (up to 512 tokens) on A100; the same kernel runs on MI300X unchanged. Limitations: falls behind at 2048+ tokens, and degrades with 64+ experts under extreme routing skew.