hugpadhugpad

amazon / FalconLite

Text Generation184171Updated Nov 17, 2023
Hugging Face

Downloads

184

Likes

171

Last updated

Nov 17, 2023

License

apache-2.0

FalconLite is a quantized version of the Falcon 40B SFT OASST-TOP1 model, capable of processing long (i.e. 11K tokens) input sequences while consuming 4x less GPU memory. By utilizing 4-bit GPTQ quantization and adapted dynamic NTK RotaryEmbedding, FalconLite achieves a balanc…

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • RefinedWeb
  • text-generation
  • custom_code

Technical specifications

ParametersNot available
Librarytransformers
LanguagesNot available
Licenseapache-2.0