Instructions to use TheBloke/llama-2-13B-Guanaco-QLoRA-GPTQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TheBloke/llama-2-13B-Guanaco-QLoRA-GPTQ with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="TheBloke/llama-2-13B-Guanaco-QLoRA-GPTQ")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("TheBloke/llama-2-13B-Guanaco-QLoRA-GPTQ") model = AutoModelForCausalLM.from_pretrained("TheBloke/llama-2-13B-Guanaco-QLoRA-GPTQ") - Notebooks
- Google Colab
- Kaggle
Error on Exlamma load in textgen
Traceback (most recent call last): File “C:\Users\xxxx\Deep\TextGen\text-generation-webui\server.py”, line 68, in load_model_wrapper shared.model, shared.tokenizer = load_model(shared.model_name, loader) File “C:\Users\xxxx\Deep\TextGen\text-generation-webui\modules\models.py”, line 79, in load_model output = load_func_maploader File “C:\Users\xxxx\Deep\TextGen\text-generation-webui\modules\models.py”, line 327, in ExLlama_HF_loader return ExllamaHF.from_pretrained(model_name) File “C:\Users\xxxx\Deep\TextGen\text-generation-webui\modules\exllama_hf.py”, line 113, in from_pretrained config.calculate_rotary_embedding_base() AttributeError: ‘ExLlamaConfig’ object has no attribute ‘calculate_rotary_embedding_base’