hugpadhugpad

CultriX / CultriX-MoE-BF16

Text Generation19B750Updated Jan 20, 2024
Hugging Face

Downloads

75

Likes

0

Last updated

Jan 20, 2024

License

apache-2.0

CultriX-MoE-BF16 is a Mixure of Experts (MoE) made with the following models using LazyMergekit: mlabonne/NeuralBeagle14-7B fblgit/UNA-dolphin-2.6-mistral-7b-dpo-laser mlabonne/Marcoro14-7B-slerp

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • mixtral
  • text-generation
  • moe
  • frankenmoe
  • merge

Technical specifications

Parameters19B
Librarytransformers
LanguagesNot available
Base modelmlabonne/NeuralBeagle14-7B
Licenseapache-2.0