hugpadhugpad

amazon / MistralLite

Text Generation528436Updated May 16, 2024
Hugging Face

Downloads

528

Likes

436

Last updated

May 16, 2024

License

apache-2.0

MistralLite is a fine-tuned Mistral-7B-v0.1 language model, with enhanced capabilities of processing long context (up to 32K tokens). By utilizing an adapted Rotary Embedding and sliding window during fine-tuning, MistralLite is able to perform significantly better on several …

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • pytorch
  • mistral
  • text-generation

Technical specifications

ParametersNot available
Librarytransformers
LanguagesNot available
Licenseapache-2.0