hugpadhugpad

Tech2wild / Nanbeige4.3-33B-W4A16-AutoRound

Text Generation6.5B3641Updated Oct 26, 2026
Hugging Face

Downloads

364

Likes

1

Last updated

Oct 26, 2026

License

apache-2.0

INT4 weight-only quant of Nanbeige/Nanbeige4.3-33B for vLLM on Ampere and newer GPUs. 20.8 GB (from 66.2 GB BF16), GPTQ format, loads with vLLM's GPTQ Marlin kernels. Fits two 24 GB cards with a 213K-token fp8 KV pool.

Tasks & library

  • text generation
  • vllm

Tags

  • vllm
  • llama
  • nanbeige
  • int4
  • w4a16
  • gptq

Technical specifications

Parameters6.5B
Libraryvllm
Languagesen, zh
Base modelNanbeige/Nanbeige4.3-33B
Licenseapache-2.0