hugpadhugpad

sophosympatheia / New-Dawn-Llama-3-70B-32K-v1.0

Text Generation71B3358Updated Jul 14, 2024
Hugging Face

Downloads

33

Likes

58

Last updated

Jul 14, 2024

License

llama3

This model is a multi-level SLERP merge of several Llama 3 70B variants. See the merge recipe below for details. I extended the context window for this model out to 32K by snagging some layers from abacusai/Smaug-Llama-3-70B-Instruct-32K using a technique similar to what I use…

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • llama
  • text-generation
  • mergekit
  • merge
  • Not-for-all-Audiences

Technical specifications

Parameters71B
Librarytransformers
LanguagesNot available
Base modelbosonai/Higgs-Llama-3-70B
Licensellama3