hugpadhugpad

shadowml / phixtral-4x2_8odd

Text Generation4.5B143Updated Jan 10, 2024
Hugging Face

Downloads

14

Likes

3

Last updated

Jan 10, 2024

License

mit

phixtral-2x28 is the first Mixure of Experts (MoE) made with two microsoft/phi-2 models, inspired by the mistralai/Mixtral-8x7B-v0.1 architecture. It performs better than each individual expert.

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • phi-msft
  • text-generation
  • moe
  • nlp
  • code

Technical specifications

Parameters4.5B
Librarytransformers
Languagesen
Licensemit