Hugging Face's logo
neuralmagic 's Collections
DeepSeek-R1-Distill Quantized
Granite 3.1 Quantization
Sparse-Llama-3.1-2of4
Vision Language Models Quantization
FP8 LLMs for vLLM
Llama-3.2 Quantization
Llama-3.1 Quantization
INT8 LLMs for vLLM
INT4 LLMs for vLLM
Sparse Foundational Llama 2 Models
Compression Papers
DeepSparse Sparse LLMs
Sparse Finetuning MPT
Compressed LLMs from the Community

INT4 LLMs for vLLM

Accurate INT4 quantized models by Neural Magic, ready for use with vLLM!