OBELICS Dataset

OBELICS
Multimodal Dataset

OBELICS (Open Bimodal Examples from Large-scale Interleaved Captioned Sources) is a large-scale multimodal web document dataset released by Hugging Face. It contains 141 million web documents, interleaved with 353 million images and 115 billion text tokens. The data is extracted from CommonCrawl, preserving the natural arrangement of text and images, used for training multimodal large language models like IDEFICS.

Get the Dataset Now
141 million web documents 353 million images 115 billion text tokens CC BY 4.0 License
OBELICS Dataset
📄
141 million
Number of web documents
🖼️
353 million
Total number of images
📝
115 billion
Text tokens
📜
CC BY 4.0
Open License Agreement

Dataset Highlights

A large-scale dataset of images and text, providing a training foundation for the next generation of multimodal large language models.

🔀

Image-Text Interleaved Format

Preserves the natural arrangement of images and text in web documents, faithfully restoring the authentic reading experience of interleaved images and text, rather than simple image-text pairing.

📊

Ultra Large Scale

Contains 141 million web documents, 353 million images, and 115 billion text tokens, making it one of the largest open-source multimodal interleaved datasets available.

🌐

Web-Native Structure

Data is extracted directly from real web pages, preserving the original document structure and contextual relationships, rather than artificially constructed image-text pairs, making it closer to natural scenarios.

🤖

Drives IDEFICS

As the core training data for Hugging Face's open-source multimodal large language model IDEFICS, it has been validated through large-scale model training in practical applications.

🛡️

Quality Selected

Processed through multiple layers of filtering pipelines, including sensitive content filtering, document quality assessment, and duplicate document removal, ensuring the data is clean and usable.

📖

Open and Reproducible

Utilizes the CC BY 4.0 license, deeply integrated with the Hugging Face Datasets library, allowing for direct streaming via API or bulk download.

Applicable Scenarios

From multimodal model training to document understanding, covering cutting-edge research and engineering implementation

🧠

Multimodal Large Model Training

Train multimodal large language models that can understand both images and text using a mixed format of text and images

📑

Document Understanding

Learn the layout and structure of real web pages to enhance the model's understanding and information extraction capabilities for complex documents

💬

Visual Question Answering

Build visual question answering models that can reason across images and text, handling complex questions that require multimodal understanding

🎯

Few-Shot Multimodal Learning

Utilize a mixed format of text and images for contextual learning, allowing the model to quickly grasp new tasks from a small number of examples

Multimodal Mixed Text and Images Large Language Model Hugging Face Web Data IDEFICS

Data Preview

The following is a structural example of a single web document from the OBELICS dataset, showcasing a mixed arrangement of text and images

JSON
{
  "document_url": "https://example.com/article",
  "content": [
    { "type": "text", "value": "Exploring the latest advancements of deep learning in natural language processing..." },
    { "type": "image", "url": "https://example.com/img1.jpg", "alt": "Transformer architecture" },
    { "type": "text", "value": "As shown in the above image, the Transformer architecture consists of encoders and decoders..." },
    { "type": "image", "url": "https://example.com/img2.jpg", "alt": "Attention mechanism" },
    { "type": "text", "value": "The attention mechanism allows the model to focus on different parts of the input sequence..." }
  ]
}

3 Steps to Get Started Quickly

Quickly integrate OBELICS into your multimodal research workflow from browsing to training

01

Browse Datasets

Browse the OBELICS datasets on the Ace Data Cloud platform to understand details such as document structure, metadata, and licensing agreements.

02

Stream or Download

Stream read or batch download data through the Hugging Face Datasets library, flexibly choosing a loading method that suits your computing resources.

03

Integrate into Training Workflow

Load the interleaved text and image documents into your multimodal training pipeline and start training the next generation of large language models that can understand the relationship between text and images.

Get the Dataset

Start Exploring OBELICS Data

141 million web documents, 353 million images, interleaved multimodal training data. Whether you are researching multimodal large models or exploring document understanding, OBELICS is the ideal data foundation.

Start Exploring OBELICS Data