hugpadhugpad

SakanaAI / RePo-OLMo2-7B-stage2-L10

Text Generation7.4B821Updated Apr 1, 2026
Hugging Face

Downloads

82

Likes

1

Last updated

Apr 1, 2026

License

apache-2.0

This repository contains a 1B parameter base model continually-pretrained from allenai/OLMo-2-1124-7B on 50B tokens using the context Re-Positioning (RePo) technique introduced in our paper RePo: Language Models with Context Re-Positioning.

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • olmo2
  • text-generation
  • en

Technical specifications

Parameters7.4B
Librarytransformers
Languagesen
Base modelallenai/OLMo-2-1124-7B
Licenseapache-2.0