EleutherAI / polyglot-ko-12.8b
Text Generation13B89085Updated Jun 7, 2023
Downloads
890
Likes
85
Last updated
Jun 7, 2023
License
apache-2.0
The model consists of 40 transformer layers with a model dimension of 5120, and a feedforward dimension of 20480. The model dimension is split into 40 heads, each with a dimension of 128. Rotary Position Embedding (RoPE) is applied to 64 dimensions of each head. The model is t…
Tasks & library
- text generation
- transformers
Tags
- transformers
- pytorch
- gpt_neox
- text-generation
- causal-lm
- ko
Technical specifications
Parameters13B
Librarytransformers
Languagesko
Licenseapache-2.0