LMDrive Dataset

LMDrive
Dataset

The LMDrive dataset consists of 64,000 instruction-sensor-control data segments collected in the CARLA simulator, each containing navigation instructions, multimodal multi-view sensor data, and control signals, lasting from 2 to 20 seconds, suitable for language-driven autonomous driving research.

Get the dataset now
64,000 data Multi-view sensors Language instructions CARLA simulator
LMDrive Dataset
📊
64,000
data segments
📷
Multi-view
multi-view sensors
⏱️
2-20s
segment duration
📜
Open
open license agreement

Dataset Highlights

A large-scale multimodal benchmark dataset for language-driven autonomous driving

💬

Language Navigation Instructions

Each data segment is accompanied by natural language navigation instructions, such as "turn right at the next intersection," supporting research on the alignment of language and driving behavior.

📷

Multi-Angle Camera Data

Includes front, left, right, and other multi-angle RGB camera image sequences, providing rich visual perception information for scene understanding.

🔵

LiDAR Point Cloud

Each frame is accompanied by high-precision LiDAR 3D point cloud data, which can be used for obstacle detection, depth estimation, and 3D spatial perception tasks.

🎮

Control Signals

Records precise control signals such as steering angle, throttle, and brake, supporting the training and evaluation of end-to-end driving models.

🏙️

CARLA Simulator Collection

Data is collected in the high-fidelity CARLA simulator, covering various weather, lighting, and traffic scenarios, safely reproducing various driving situations.

🛣️

Diverse Driving Scenarios

Covers various road types and traffic scenarios, including urban roads, intersections, roundabouts, and highways, enhancing model generalization capabilities.

Applicable Scenarios

From academic research to industrial prototype validation, covering the core directions of autonomous driving

🗣️

Language-Guided Driving

Training driving models that can understand natural language navigation instructions, achieving human-machine language interactive autonomous driving

🚗

End-to-End Autonomous Driving

End-to-end learning directly from sensor input to control output, exploring driving strategies without intermediate representations

🔗

Multimodal Fusion

Integrating various modal data such as camera images, LiDAR point clouds, and language instructions to enhance perception and decision-making capabilities

📋

Instruction Following

Evaluating the model's understanding and execution capabilities of human navigation instructions, promoting the development of interactive autonomous driving systems

Autonomous Driving Language-Driven CARLA Multimodal End-to-End

Data Preview

The following is a typical structure of a single data segment, including instructions, sensor paths, and control signals

JSON
{
  "instruction": "Turn right at the next intersection.",
  "segment_id": "route_00042_seg_0015",
  "duration": 8.2,
  "sensors": {
    "camera_front": "data/route_00042/seg_0015/rgb_front/*.png",
    "camera_left": "data/route_00042/seg_0015/rgb_left/*.png",
    "camera_right": "data/route_00042/seg_0015/rgb_right/*.png",
    "lidar": "data/route_00042/seg_0015/lidar/*.npy"
  },
  "control": [
    { "frame": 0, "steering": 0.00, "throttle": 0.60, "brake": 0.00 },
    { "frame": 10, "steering": 0.35, "throttle": 0.45, "brake": 0.00 },
    { "frame": 20, "steering": 0.72, "throttle": 0.30, "brake": 0.00 },
    { "frame": 30, "steering": 0.50, "throttle": 0.55, "brake": 0.00 },
    { "frame": 40, "steering": 0.05, "throttle": 0.65, "brake": 0.00 }
  ],
  "weather": "ClearNoon",
  "town": "Town03"
}

3 Steps to Get Started Quickly

From browsing to loading, you can start your autonomous driving research in just a few minutes

01

Browse the Dataset

View the details of the LMDrive dataset on the Ace Data Cloud platform, and learn about metadata such as data structure, scene distribution, and licensing agreements.

02

Download Data

Download 64,000 instruction-sensor-control data segments, including multi-view camera images, LiDAR point clouds, and control signals.

03

Load and Train

Use the official LMDrive toolkit to load the data and start training and evaluating the language-driven autonomous driving model.

Get the Dataset

Start Exploring the LMDrive Dataset

A large-scale language-driven autonomous driving dataset with open licensing, available for immediate download. Whether you are an autonomous driving researcher or a multimodal AI developer, this dataset is worth trying.

Get the Dataset Now