FAVDBench Dataset

FAVDBench
Dataset

FAVDBench (Fine-grained Audible Video Description Benchmark) is a fine-grained audio-visual description benchmark dataset proposed at CVPR 2023, aimed at providing detailed textual descriptions for audible videos that include object appearance, spatial location, action information, and sound descriptions.

Get the Dataset Now

11,000+ videos 5 types of descriptions CVPR 2023 OpenNLPLab

🎬

11,000+

Number of Videos

📝

Types of Descriptions

🏆

CVPR 2023

Top Conference

📜

Open

Open License Agreement

Dataset Highlights

A fine-grained description benchmark for audio-visual understanding, pushing the frontier of multimodal research

🔊

Audio-Visual Fusion

Covering both visual and auditory information, it is one of the few datasets that incorporates audio descriptions into video description benchmarks, supporting cross-modal research.

🎯

Fine-Grained Descriptions

Provides fine-grained text annotations across five dimensions: Appearance, Spatial, Temporal, Action, and Audio.

🏷️

Multimodal Annotations

Each video contains multi-dimensional human annotation information, with high annotation quality, suitable for training and evaluating multimodal generation models.

📊

Academic Benchmark

Proposed and established by a CVPR 2023 paper, widely cited in academia, it is the standard evaluation benchmark for audio-visual description tasks.

🌐

Diverse Content

The videos cover a variety of scenes and themes, including natural scenes, human activities, animal behaviors, etc., ensuring a comprehensive assessment of model generalization capabilities.

🔧

Openly Available

The dataset is released under an open license, allowing researchers to freely download and use it, lowering the barriers for academic research and industrial applications.

Applicable Scenarios

Empowering audio and video understanding technology from academic research to industrial applications

📹

Video Description Generation

Train and evaluate video description generation models to automatically generate multidimensional natural language descriptions for videos

🔊

Audio and Video Understanding

Research the joint understanding of visual and auditory information, exploring cross-modal semantic alignment and fusion methods

🧠

Multimodal Research

Provide high-quality training and evaluation data for visual-language-audio three-modal pre-trained models

💬

Video Subtitle Generation

Develop automatic video subtitle systems to enhance the accessibility and retrievability of video content

Video Description Audio and Video Understanding Multimodal CVPR Benchmarking

Data Preview

The following are annotation examples from the FAVDBench dataset, containing refined descriptions across five dimensions

JSON

{
  "video_id": "video_00123",
  "descriptions": {
    "appearance": "A brown dog with floppy ears and a red collar stands on green grass.",
    "spatial": "The dog is positioned in the center of the frame with trees in the background.",
    "temporal": "The video starts with the dog sitting, then it stands up and begins to walk.",
    "action": "The dog wags its tail, barks twice, and runs toward the camera.",
    "audio": "Birds chirping in the background, followed by two loud barks and rustling grass."
  },
  "duration": 8.5,
  "split": "train"
}

3 Steps to Get Started Quickly

From browsing to usage, you can start your multimodal research in just a few minutes

Browse Datasets

View dataset details on the Ace Data Cloud platform to understand metadata such as annotation format, data scale, and usage licenses.

Download Data

Download video files and JSON annotation data; the dataset provides standard splits for training, validation, and testing sets.

Load and Use

Use json.load() to load annotation data and start training and evaluating multimodal models with video processing libraries.

Get the Dataset

Start Exploring the FAVDBench Dataset

CVPR 2023 Fine-grained Audio-Video Description Benchmark, open license, download now. Whether you are a multimodal researcher or a video understanding engineer, this dataset is worth a try.

Get the Dataset Now

Dataset