hugpadhugpad

mistral-community / Mixtral-8x22B-v0.1-4bit

Text Generation141B19555Updated Apr 10, 2024
Hugging Face

Downloads

195

Likes

55

Last updated

Apr 10, 2024

License

apache-2.0

Model details: - 🧠 ~176B params, ~44B active during inference - 🪟 65K context window - 🕵🏾‍♂️ 8 experts, 2 per token - 🤓 32K vocab size - ✂️ Similar tokenizer as 7B

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • mixtral
  • text-generation
  • moe
  • fr
  • it

Technical specifications

Parameters141B
Librarytransformers
Languagesfr, it, de, es, en
Licenseapache-2.0