nisten / qwenv2-7b-inst-imatrix-gguf
Other2053Updated Jun 16, 2024
Downloads
205
Likes
3
Last updated
Jun 16, 2024
License
apache-2.0
These are a whole bunch of conversions of qwen7b v2 in an attempt to fix the reduced performance while quantizing. The bf16 versions will NOT work with apple GPUs but will work with most cpus and newer nvidia cards (older ones like 1080 series don't support bf16 inference well).
Tags
- gguf
- imatrix
- conversational
Technical specifications
ParametersNot available
LibraryNot available
LanguagesNot available
Base modelQwen/Qwen2-7B-Instruct
Licenseapache-2.0
