nisten / qwenv2-7b-inst-imatrix-gguf

Other2053Updated Jun 16, 2024

Downloads

205

Likes

3

Last updated

Jun 16, 2024

License

apache-2.0

These are a whole bunch of conversions of qwen7b v2 in an attempt to fix the reduced performance while quantizing. The bf16 versions will NOT work with apple GPUs but will work with most cpus and newer nvidia cards (older ones like 1080 series don't support bf16 inference well).

Tags

  • gguf
  • imatrix
  • conversational

Technical specifications

ParametersNot available
LibraryNot available
LanguagesNot available
Licenseapache-2.0