Quazim0t0 / Byrne-86M-Base
Text Generation97M1554Updated Sep 20, 2026
Downloads
155
Likes
4
Last updated
Sep 20, 2026
License
apache-2.0
Base of the Byrne family. Distilled step-4000 checkpoint. ~86M SpikeWhaleLM from scratch - MLA, n-gram engram, hash-lookup, hyper-connections, HRM refine, MTP. Custom ChatML-aware tokenizer. I use this as a general base to keep pretraining / SFT.
Tasks & library
- text generation
- transformers
Tags
- transformers
- spike_whale
- feature-extraction
- text-generation
- small-models
- mla
Technical specifications
Parameters97M
Librarytransformers
Languagesen
Licenseapache-2.0