IFM / K2-Horizon-MoVA-36B-A4B
Text Generation37B13.6K430Updated Oct 14, 2026
Downloads
13.6K
Likes
430
Last updated
Oct 14, 2026
License
apache-2.0
K2-Horizon-MoVA-36B-A4B is the sparse member of the K2-Horizon family: a Mixture-of-Experts model with Mixture-of-Values attention (MoVA) that stores 36B parameters and runs 4B per token. We have released the checkpoints, along with the data and the training code.
Tasks & library
- text generation
- transformers
Tags
- transformers
- k2_horizon
- text-generation
- k2-horizon
- 36b
- mova
Technical specifications
Parameters37B
Librarytransformers
Languagesen
Licenseapache-2.0