TinyLlama / TinyLlama-1.1B-intermediate-step-1431k-3T
Text Generation1.1B50.3K194Updated Sep 27, 2024
Downloads
50.3K
Likes
194
Last updated
Sep 27, 2024
License
apache-2.0
The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.
Tasks & library
- text generation
- transformers
Tags
- transformers
- pytorch
- llama
- text-generation
- en
- model-index
Technical specifications
Parameters1.1B
Librarytransformers
Languagesen
Licenseapache-2.0