Transformers How to use prasannaJagadesh/marlin-2B-GPTQ-4BITS with Transformers:
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM
processor = AutoProcessor.from_pretrained("prasannaJagadesh/marlin-2B-GPTQ-4BITS", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("prasannaJagadesh/marlin-2B-GPTQ-4BITS", trust_remote_code=True, device_map="auto") 
