Hugging Face's logo

openbmb
/
MiniCPM-Llama3-V-2_5-int4

Visual Question Answering
Transformers
Safetensors
minicpmv
feature-extraction
custom_code
4-bit precision
bitsandbytes
Model card Files
xet
 Community
9

report for local try run

#2
by kennyx - opened
Discussion
kennyx

I'm trying to run the model locally. for both MAC M3 chip and windows 2080Ti
the same error occured:
"""
ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes
"""
I've tried to re-install the latest version. unfortunate, it's still happy.
I'm not sure how to fix this.
harshitface2003

I am facing the same error today, have you fixed it?
stargram

I encountered the same error, and I resolved it by following these steps:

  1. Execute the command python -c 'import torch; print(torch.version.cuda)'. If it returns None, this indicates that you have installed the CPU-only version of PyTorch without the necessary CUDA runtime.
  2. To rectify this, use the installation commands provided here, ensuring that the installation log confirms the selection of the appropriate binary.
  3. Ensure that the torch version is updated from 2.1.2 to 2.1.2+cu121, and it should works.

Good luck.
Reference

Sign up or log in to comment