hugpadhugpad

grimjim / kukulemon-32K-7B

Text Generation7.2B482Updated Apr 27, 2024
Hugging Face

Downloads

48

Likes

2

Last updated

Apr 27, 2024

License

cc-by-nc-4.0

This is a proof of concept a merge capable of functional 32K context length while being derived from kukulemon-7B. The functioning 32K context window has been folded in via a merger of Mistral 7B v0.2 models. SLERP merge appears to be viable, but DARE-TIES merge risks producin…

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • mistral
  • text-generation
  • mergekit
  • merge

Technical specifications

Parameters7.2B
Librarytransformers
LanguagesNot available
Base modelgrimjim/Mistral-7B-Instruct-demi-merge-v0.2-7B
Licensecc-by-nc-4.0