hugpadhugpad

ibivibiv / hydra-moe-120b

Text Generation114B181Updated Feb 2, 2024
Hugging Face

Downloads

18

Likes

1

Last updated

Feb 2, 2024

License

apache-2.0

This is a 4x34b MOE model. I have been expirementing with various gate settings and models. This is yet another run with a different set of models on the same gate definitions.

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • mixtral
  • text-generation
  • moe
  • moerge
  • conversational

Technical specifications

Parameters114B
Librarytransformers
Languagesen
Licenseapache-2.0