artifex-rp-orpheus-llama-3.1-8b-lora

LoRA adapter for meta-llama/Llama-3.1-8B-Instruct. For most users the merged model at darthcrawl/artifex-rp-orpheus-llama-3.1-8b is easier to use.

Apply

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_id = "meta-llama/Llama-3.1-8B-Instruct"
adapter_id = "darthcrawl/artifex-rp-orpheus-llama-3.1-8b-lora"

tok = AutoTokenizer.from_pretrained(base_id, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)

Sampling settings

Parameter	Value
temperature	0.85
top_p	0.95
repetition_penalty	1.1

System prompt

Write in vivid, restrained, intelligent prose. Concrete sensory detail.
Never pretentious. Tension carries more weight than description.
Match the user's energy and length.

License

Same license as the base model (meta-llama/Llama-3.1-8B-Instruct). Apache 2.0 for the training pipeline.