Llama-3.2 Quantization - a neuralmagic Collection

neuralmagic 's Collections

Sparse-Llama-3.1-2of4

Vision Language Models Quantization

FP8 LLMs for vLLM

Llama-3.2 Quantization

Llama-3.1 Quantization

INT8 LLMs for vLLM

INT4 LLMs for vLLM

Sparse Foundational Llama 2 Models

Compression Papers

DeepSparse Sparse LLMs

Sparse Finetuning MPT

Compressed LLMs from the Community

Llama-3.2 Quantization

updated Sep 26, 2024

Llama 3.2 models quantized by Neural Magic

neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2, 2024 • 4.55k • 17
neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2, 2024 • 3.31k • 8
neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9, 2024 • 2.16k • 2
neuralmagic/Llama-3.2-3B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9, 2024 • 1.39k • 2
neuralmagic/Llama-3.2-1B-Instruct-quantized.w8a8

Text Generation • Updated Oct 16, 2024 • 3.83k • 7
neuralmagic/Llama-3.2-3B-Instruct-quantized.w8a8

Text Generation • Updated Oct 16, 2024 • 6.24k • 1
neuralmagic/Llama-3.2-1B-Instruct-FP8

Text Generation • Updated Oct 16, 2024 • 6.58k • 2
neuralmagic/Llama-3.2-3B-Instruct-FP8

Text Generation • Updated Oct 16, 2024 • 16.1k • 3
neuralmagic/Llama-3.2-1B-FP8

Updated Oct 9, 2024 • 12