Hybrid Video Summarization: Blending Rule-Based Linguistic Filtering with BART-Based Text Summarization and BLIP-Based Visual Captioning
Keywords:
Video Summarization, Rule-Based Linguistic Scoring, Transformer Models, Multimodal Fusion, Embedding ConcatenationAbstract
As online multimedia grows rapidly, long multimedia videos often overwhelm users, and thus there is a demand for effective ways to summarise them. The current solutions also involve trade-offs: rule-based solutions are both fast and understandable, but they lose semantic subtext, whereas transformer-based systems like BART (Bidirectional and Autoregressive Transformers) and BLIP (Bootstrapping Language-Image Pre-training) can generate high-quality abstractive summaries, but are expensive and inscrutable. This paper presents a hybrid model, which is a blend of deterministic linguistic scoring with transformer-based refining and multimodal fusion. Rule-based importance scores based on positional cues, named entities, TF-IDF relevance, and redundancy suppression are first used to filter the transcript segments. Key frames are hedged using BLIP, and the textual and visual embeddings are merged through concatenation and sent to BART to be summarised in abstraction. The results of experiments based on the TVSum dataset indicate that the hybrid model is better than both pure rule-based and pure transformer baselines, with ROUGE-1: 0.52, ROUGE-2: 0.35, F1-score: 0.65, and smaller transformer input sizes, with nearly 3 times faster inference. The framework further offers explainable selection criteria for selected segments, closing the gap between interpretability and semantic richness. The developed approach provides a realistic, effective, and clear multimodal video summarization solution, which can be extended to domain-specific data, real-time streaming, and massive multimodal language models.
