Text Generation
PyTorch
Safetensors
llama
GPTQ
8bit
128g
1B
facebook
meta
llama-3
conversational
8-bit precision
gptq
How to use from
vLLMUse Docker
docker model run hf.co/adriabama06/Llama-3.2-1B-Instruct-GPTQ-8bit-128gQuick Links
GPTQ Version of meta-llama/Llama-3.2-1B: 8 bit, 128 groupsize
✅ Tested on vLLM
- Downloads last month
- 8
Model tree for adriabama06/Llama-3.2-1B-Instruct-GPTQ-8bit-128g
Base model
meta-llama/Llama-3.2-1B-Instruct
Install from pip and serve model
# Install vLLM from pip: pip install vllm# Start the vLLM server: vllm serve "adriabama06/Llama-3.2-1B-Instruct-GPTQ-8bit-128g"# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "adriabama06/Llama-3.2-1B-Instruct-GPTQ-8bit-128g", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'