Llama.cpp : Split Mode Tensor Fix Incoming?
r/LocalLLaMA
•
Generative AI
AI Hardware
Open Source AI
Appears thay have been cooking and we might see a fix soon released for crashes on split mode tensor Multi-gpu folks keep watch - ( In my tests SM Tensor has a ~35% uplift in TG over Layer but ofc crashes every 90-120 minutes due to vram exhaustion this fix is supposed to stop that ) submitted by /u/Bulky-Priority6824 [link] [comments]