AI RESEARCH
OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants
arXiv CS.CL
•
We introduce OmniInteract, a streaming benchmark for real-time omnimodal large language models evaluated through native online inference over audio-visual streams. Unlike offline video understanding or text-prompted streaming QA, OmniInteract preserves the original audio-visual stream and requires models to process it online, without access to future content. User queries and ambient sounds are embedded in the audio track, requiring models to detect multimodal triggers, decide when to respond, a