hugpadhugpad

RedHatAI / Llama-3.2-3B-Instruct-FP8-dynamic

Text Generation3.6B354.8K3Updated Oct 9, 2024
Hugging Face

Downloads

354.8K

Likes

3

Last updated

Oct 9, 2024

License

llama3.2

Quantized version of Llama-3.2-3B-Instruct. It achieves an average score of 50.88 on a subset of task from the OpenLLM benchmark (version 1), whereas the unquantized model achieves 51.70.

Tasks & library

  • text generation

Tags

  • llama
  • fp8
  • vllm
  • text-generation
  • conversational
  • en

Technical specifications

Parameters3.6B
LibraryNot available
Languages8 languages
Base modelmeta-llama/Llama-3.2-3B-Instruct
Licensellama3.2