hugpadhugpad

jtatman / Tiny-Llama-Llama-Dolphin-laser-1b-moe

Text Generation3.4B382Updated Feb 25, 2024
Hugging Face

Downloads

38

Likes

2

Last updated

Feb 25, 2024

License

apache-2.0

Tiny-Llama-Llama-Dolphin-laser-1b-moe is a Mixure of Experts (MoE) made with the following models using LazyMergekit: TinyLlama/TinyLlama-1.1B-intermediate-step-715k-1.5T cognitivecomputations/TinyDolphin-2.8.2-1.1b-laser cognitivecomputations/TinyDolphin-2.8.1-1.1b TinyLlama/…

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • mixtral
  • text-generation
  • moe
  • frankenmoe
  • merge

Technical specifications

Parameters3.4B
Librarytransformers
LanguagesNot available
Base modelTinyLlama/TinyLlama-1.1B-intermediate-step-715k-1.5T
Licenseapache-2.0