TinyStories Dataset

TinyStories
Dataset

A dataset of 2.14 million synthetic short stories generated by Microsoft Research using GPT-3.5 and GPT-4, designed for training small language models. It demonstrates that models with only 10 million parameters can generate fluent and coherent English stories. MIT License.

Get the dataset now
2.14 million stories Generated by GPT-3.5/4 MIT License Microsoft Research
📚
2.14M
Total number of stories
🤖
GPT-3.5/4
Generating model
📜
MIT
Open source license
🔬
Microsoft
Produced by Microsoft Research

Dataset Highlights

A high-quality synthetic story dataset designed specifically for training small language models

🎯

Focus on Small Models

Designed for training small language models (10M-100M parameters), proving that with the right data, small models can generate fluent and coherent English text.

✨

Synthetic High Quality

Generated using GPT-3.5 and GPT-4, each story is guided by carefully designed prompts to ensure narrative quality and linguistic diversity.

🧒

Simple Vocabulary

Uses vocabulary and grammatical structures that 3-4 year old children can understand, reducing language complexity to make it easier for small models to learn the basic patterns of language.

📖

Coherent Narratives

Each story contains a complete narrative structure—beginning, development, and ending, helping the model learn the logical reasoning abilities of story-telling.

🌈

Diverse Themes

Covers a rich variety of themes such as friendship, adventure, animals, family, and nature, ensuring content diversity through random combinations of keywords and story features.

🏛️

MIT Open Source

Adopts the MIT license, allowing free commercial and academic use without worrying about licensing restrictions, suitable for various research and application scenarios.

Applicable Scenarios

From cutting-edge research to educational applications, empowering various directions of small language models

🧠

Small LM Training

Train small language models with 10M-100M parameters, achieving smooth text generation capabilities under resource-constrained conditions

🔬

Language Model Research

Research the Scaling Laws of language models, the impact of data quality on model performance, and the minimum scale threshold for emergent capabilities

🎓

Educational AI

Develop AI applications for children's education, such as automatic story generation, reading assistance, and language learning tools

⚡

Efficient Model Development

Rapidly iterate model architectures and training strategies on consumer-grade hardware, lowering the computational threshold for NLP research

NLP stories small-models synthetic education

API Call Example

Quickly access the TinyStories dataset via API

BASH
curl -X GET "https://api.acedata.cloud/datasets/tinystories" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json"
# Response
{
  "name": "TinyStories",
  "description": "2.14M synthetic short stories generated by GPT-3.5/GPT-4",
  "license": "MIT",
  "source": "Microsoft Research",
  "total_stories": 2140000,
  "format": "text",
  "language": "en"
}

3 Steps to Get Started Quickly

From browsing to training, you can start your small model research in just a few minutes

01

Browse the Dataset

View the details of the TinyStories dataset on the Ace Data Cloud platform, learn about the data scale, generation method, and licensing agreement.

02

Download the Data

Obtain 2.14 million synthetic short stories via the API, with a simple data format that is ready to use, requiring no additional cleaning.

03

Load and Train

Use datasets.load_dataset() to load the data and start training your small language model.

Get the Dataset

Start Exploring the TinyStories Dataset

2.14 million high-quality synthetic stories, MIT open-source license, available immediately. Whether you are an NLP researcher or an AI education developer, this dataset is an ideal choice for training small language models.

Get the Dataset Now