RedHatAI / Llama-3.3-70B-Instruct-FP8-dynamic

This model was obtained by quantizing activation and weights of Llama-3.3-70B-Instruct to FP8 data type. This optimization reduces the number of bits used to represent weights and activations from 16 to 8, reducing GPU memory requirements (by approximately 50%) and increasing …