hugpadhugpad

SakanaAI / RePo-OLMo2-1B-stage2-L5

Text Generation1.5B1432Updated Dec 18, 2025
Hugging Face

Downloads

143

Likes

2

Last updated

Dec 18, 2025

License

apache-2.0

This repository contains a 1B parameter base model continually-pretrained from allenai/OLMo-2-0425-1B on 50B tokens using the context Re-Positioning (RePo) technique introduced in our paper RePo: Language Models with Context Re-Positioning.

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • olmo2
  • text-generation
  • en

Technical specifications

Parameters1.5B
Librarytransformers
Languagesen
Base modelallenai/OLMo-2-0425-1B
Licenseapache-2.0