LongInsightBench: A Comprehensive Benchmark for Evaluating Omni-Modal Models on Human-Centric Long-Video Understanding

We introduce $\mathbf{\text{LongInsightBench}}$, the first benchmark designed to assess models’ ability to understand long videos, with a focus on human language, viewpoints, actions, and other contextual elements, while integrating $\mathbf{\text{visual,}}\mathbf{\text{audio,}}\mathbf{\text{and}}\mathbf{\text{text}}$ modalities. Our benchmark excels in three key areas: $\mathbf{\text{a)}}\mathbf{\text{Long-Duration,}}\mathbf{\text{Information-Dense}}\mathbf{\text{Videos:}}$ We carefully select approximately 1,000 videos from open-source datasets FineVideo based on duration limit and the information density of both visual and audio modalities, focusing on content like lectures, interviews, and vlogs, which contain rich language elements. $\mathbf{\text{b)}}\mathbf{\text{Diverse}}\mathbf{\text{and}}\mathbf{\text{Challenging}}\mathbf{\text{Task}}\mathbf{\text{Scenarios:}}$ We have designed six challenging task scenarios, including both Intra-Event and Inter-Event Tasks. $\mathbf{\text{c)}}\mathbf{\text{Rigorous}}\mathbf{\text{and}}\mathbf{\text{Comprehensive}}\mathbf{\text{Quality}}\mathbf{\text{Assurance}}\mathbf{\text{Pipelines:}}$ We have developed a three-step, semi-automated data quality assurance pipeline to ensure the difficulty and validity of the synthesized questions and answer options. Based on LongInsightBench, we designed a series of experiments. Experimental results shows that Omni-modal models(OLMs) still face challenge in tasks requiring precise temporal localization (T-Loc) and long-range causal inference (CE-Caus). Extended experiments reveal the information loss and processing bias in multi-modal fusion of OLMs. Our dataset and code is available at https://anonymous.4open.science/r/LongInsightBench-910F/.