hugpadhugpad

CultriX / MoNeuTrix-MoE-4x7B

Text Generation24B750Updated Mar 24, 2024
Hugging Face

Downloads

75

Likes

0

Last updated

Mar 24, 2024

License

apache-2.0

MoNeuTrix-MoE-4x7B is a Mixture of Experts (MoE) made with the following models using LazyMergekit: CultriX/MonaTrix-v4 mlabonne/OmniTruthyBeagle-7B-v0 CultriX/MoNeuTrix-7B-v1 paulml/OmniBeagleSquaredMBX-v3-7B

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • mixtral
  • text-generation
  • moe
  • frankenmoe
  • merge

Technical specifications

Parameters24B
Librarytransformers
LanguagesNot available
Base modelCultriX/MonaTrix-v4
Licenseapache-2.0