KnutJaegersberg / 2-bit-LLMs

This is a model collection of mostly larger LLMs quantized to 2 bit with the novel quip# inspired approach in llama.cpp Sometimes both xs and xxs are available. Note that for some larger models, like Qwen-72b based models, the context length might be too large for most GPUs, s…