hugpadhugpad

shadowml / phixtral-3x2_8

Text Generation6.1B130Updated Jan 21, 2024
Hugging Face

Downloads

13

Likes

0

Last updated

Jan 21, 2024

License

mit

phixtral-3x28 is the first Mixure of Experts (MoE) made with two microsoft/phi-2 models, inspired by the mistralai/Mixtral-8x7B-v0.1 architecture. It performs better than each individual expert.

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • phi-msft
  • text-generation
  • moe
  • nlp
  • code

Technical specifications

Parameters6.1B
Librarytransformers
Languagesen
Licensemit