Instructions to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1", dtype="auto")

llama-cpp-python

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1",
	filename="gguf/q2_k_gguf/Qwen3.5-0.8B.Q2_K.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Use Docker

docker model run hf.co/Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

LM Studio
Jan

vLLM

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

SGLang

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Ollama:
```
ollama run hf.co/Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
```

Unsloth Studio

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 to start chatting

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Docker Model Runner:
```
docker model run hf.co/Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
```

Lemonade

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Run and chat with the model

lemonade run user.qwen3.5-0.8b-unslop-good-lora-v1-Q4_K_M

List all available models

lemonade list

qwen3.5-0.8b-unslop-good-lora-v1

Smallest and cheapest lane in the Unslop family.

This is a Qwen 3.5 0.8B fine-tune for rewrite-style cleanup: take AI-sounding prose, rewrite it into cleaner and more natural text, and keep the meaning intact.

If you want the best quality in this family, start with 4B. If you want the lightest pilot or the lowest-cost baseline, this is the one.

Quick links

Hub repo: Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1
GGUF files: gguf/

Recommended downloads

If you just want the model files, the GGUF folder now contains only the final quantized artifacts:

gguf/q2_k_gguf/Qwen3.5-0.8B.Q2_K.gguf
gguf/q4_k_m_gguf/Qwen3.5-0.8B.Q4_K_M.gguf
gguf/q6_k_gguf/Qwen3.5-0.8B.Q6_K.gguf
gguf/q8_0_gguf/Qwen3.5-0.8B.Q8_0.gguf

Format	Best for	Notes
`q6_k`	Default local use	Best balance for the 0.8B lane
`q4_k_m`	Low-VRAM use	Smaller and faster, with a quality drop
`q8_0`	Highest quality	Largest file size, most faithful among the GGUFs
`q2_k`	Tiny / fastest	Emergency fallback only

My practical recommendation: download q6_k first, then keep q4_k_m around if you need a smaller fallback.

What this model is for

Use this model if you want:

a cheap rewrite baseline
a fast pilot before scaling to 2B / 4B
a lightweight deployment candidate for simple cleanup tasks
a comparison point for judging whether a larger model is worth the extra cost

What it is not for

Be careful with this lane if you need:

strict factual preservation on long inputs
the strongest style fidelity in the family
production-grade rewriting with minimal drift

In this family, 0.8B is the roughest member. It can do real rewrites, but it is clearly less stable than 2B and 4B.

How it was trained

Base model: Qwen/Qwen3.5-0.8B
Training path: Transformers / TRL / PEFT fine-tuning on Hugging Face Jobs
Dataset: N8Programs/unslop-good
Rows used: 1000 (full training split)
Objective: direct rewrite / style cleanup

Training shape

hardware: A10G 24GB (a10g-large)
max_seq_length: 2048
num_train_epochs: 2
batch_size: 1
gradient_accumulation_steps: 1
learning_rate: 1e-4
scheduler: cosine
warmup_steps: 50
LoRA rank: 8
LoRA alpha: 20
LoRA dropout: 0.0
4-bit loading
bf16 training

Deployment

This run is deployment-backed and live on Modal.

Status: healthy
Base model: Qwen/Qwen3.5-0.8B
Output repo: Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1
Mode: standard Unsloth / HF adapter loading
Endpoint style: rewrite API
Thinking mode: disabled

Live endpoint generation defaults used by the app:

enable_thinking = false
temperature = 0.7
top_p = 0.8
top_k = 20
min_p = 0.0
presence_penalty = 1.5
repetition_penalty = 1.0

Evaluation snapshot

Short eval summary only (full long-form input/output removed due poor quality).

Short sanity sample:

Rewrite this short-form promo hook to sound less generic and more human.

Full observed output:

The old ad said everything about being the best in town, but that's just a cliché. I'm not trying to be the king of the city. I'm just here to show you how we actually do things. If you want to see the real stuff, come by and let me show you what makes us special. Let's get started.

Held-out fiction sample (input):

The wind rose all at once, as if the mountain itself had inhaled. She stood at the edge of the pass, cloak snapping like a torn banner behind her, staring down at the valley where the last lights were going out one by one. Each flame extinguished felt deliberate—an answer to a question she had been too afraid to ask. Somewhere below, bells began to ring, not in warning, but in mourning. They already knew. “Don’t,” he said, breathless, reaching for her arm. His fingers brushed fabric, then air. “If you cross that line, there’s no coming back.” She didn’t turn. The cold burned her lungs, sharpened her thoughts. “There was never a way back,” she replied softly. “Only a way forward you were willing to pretend didn’t exist.” Thunder rolled, low and ancient. For a moment, the sky fractured into white fire, and in that instant he saw her face—resolved, terrified, incandescent with purpose. Not the girl he had sworn to protect, but the woman the world had demanded she become. “You’ll die,” he said. She smiled then, finally facing him, eyes bright with unshed tears. “So will everyone,” she answered. “The difference is what we leave burning when we’re gone.” She stepped forward. The mountain exhaled.

Judgment

Blunt judgment: this is the weakest lane in the family, but it is still a real model, not a toy. It can rewrite long prose in a coherent way, just not as reliably or as naturally as the larger lanes.

Use it when you care more about cost and latency than final quality.

Family position

0.8B: cheapest pilot, roughest output
2B: better balance of fidelity and fluency
4B: strongest default candidate in the small-model set
9B: useful retrain, but still not the safest long-form rewrite choice versus 4B

Training loss vs progress

See the normalized family comparison plot below.

Bottom line

If you want the lightest Unslop lane, this is the one to grab. If you want higher rewrite quality, scale up to the larger lanes (9B and especially 30B-A3B for quality-first use).

Downloads last month: 19

Safetensors

Model size

0.9B params

Tensor type

F32

BF16

Model tree for Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1

Base model

Qwen/Qwen3.5-0.8B-Base

Finetuned

Qwen/Qwen3.5-0.8B

Adapter

(157)

this model