How to use prasannaJagadesh/marlin-2B-GPTQ-4BITS with Transformers:
# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("prasannaJagadesh/marlin-2B-GPTQ-4BITS", trust_remote_code=True) model = AutoModelForMultimodalLM.from_pretrained("prasannaJagadesh/marlin-2B-GPTQ-4BITS", trust_remote_code=True, device_map="auto")
