EleutherAI / polyglot-ko-5.8b
Text Generation6B90266Updated Jun 7, 2023
Downloads
902
Likes
66
Last updated
Jun 7, 2023
License
apache-2.0
The model consists of 28 transformer layers with a model dimension of 4096, and a feedforward dimension of 16384. The model dimension is split into 16 heads, each with a dimension of 256. Rotary Position Embedding (RoPE) is applied to 64 dimensions of each head. The model is t…
Tasks & library
- text generation
- transformers
Tags
- transformers
- pytorch
- gpt_neox
- text-generation
- causal-lm
- ko
Technical specifications
Parameters6B
Librarytransformers
Languagesko
Licenseapache-2.0