hugpadhugpad

jsfs11 / MixtureofMerges-MoE-4x7b-v3

Text Generation24B890Updated Mar 4, 2024
Hugging Face

Downloads

89

Likes

0

Last updated

Mar 4, 2024

License

apache-2.0

MixtureofMerges-MoE-4x7b-v3 is a Mixure of Experts (MoE) made with the following models using LazyMergekit: jsfs11/RandomMergeNoNormWEIGHTED-7B-DARETIES senseable/WestLake-7B-v2 mlabonne/OmniBeagle-7B vanillaOVO/supermariov3

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • mixtral
  • text-generation
  • moe
  • frankenmoe
  • merge

Technical specifications

Parameters24B
Librarytransformers
LanguagesNot available
Base modeljsfs11/RandomMergeNoNormWEIGHTED-7B-DARETIES
Licenseapache-2.0