Instructions to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1", dtype="auto")

llama-cpp-python

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1",
	filename="gguf/q2_k_gguf/Qwen3.5-0.8B.Q2_K.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Use Docker

docker model run hf.co/Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

LM Studio
Jan

vLLM

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

SGLang

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Ollama:
```
ollama run hf.co/Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
```

Unsloth Studio

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 to start chatting

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Docker Model Runner:
```
docker model run hf.co/Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M
```

Lemonade

How to use Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1:Q4_K_M

Run and chat with the model

lemonade run user.qwen3.5-0.8b-unslop-good-lora-v1-Q4_K_M

List all available models

lemonade list

Oysiyl commited on Apr 8

Commit

0853bc4

verified ·

1 Parent(s): 5a74995

docs: add model card metadata and usage notes

Browse files

Files changed (1) hide show

README.md +33 -134

README.md CHANGED Viewed

@@ -11,143 +11,42 @@ tags:
 - rewriting
 - style-transfer
 - unslop
-- text-generation
 ---
 # qwen3.5-0.8b-unslop-good-lora-v1
-Smallest and cheapest lane in the Unslop family.
-This is a Qwen 3.5 0.8B fine-tune for rewrite-style cleanup: take AI-sounding prose, rewrite it into cleaner and more natural text, and keep the meaning intact.
-If you want the best quality in this family, start with 4B. If you want the lightest pilot or the lowest-cost baseline, this is the one.
-## Quick links
-- Hub repo: [Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1](https://huggingface.co/Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1)
-- GGUF files: [gguf/](./tree/main/gguf)
-## Recommended downloads
-If you just want the model files, the GGUF folder now contains only the final quantized artifacts:
-- `gguf/q2_k_gguf/Qwen3.5-0.8B.Q2_K.gguf`
-- `gguf/q4_k_m_gguf/Qwen3.5-0.8B.Q4_K_M.gguf`
-- `gguf/q6_k_gguf/Qwen3.5-0.8B.Q6_K.gguf`
-- `gguf/q8_0_gguf/Qwen3.5-0.8B.Q8_0.gguf`
-| Format | Best for | Notes |
-| --- | --- | --- |
-| `q6_k` | Default local use | Best balance for the 0.8B lane |
-| `q4_k_m` | Low-VRAM use | Smaller and faster, with a quality drop |
-| `q8_0` | Highest quality | Largest file size, most faithful among the GGUFs |
-| `q2_k` | Tiny / fastest | Emergency fallback only |
-My practical recommendation: download `q6_k` first, then keep `q4_k_m` around if you need a smaller fallback.
-## What this model is for
-Use this model if you want:
-- a cheap rewrite baseline
-- a fast pilot before scaling to 2B / 4B
-- a lightweight deployment candidate for simple cleanup tasks
-- a comparison point for judging whether a larger model is worth the extra cost
-## What it is not for
-Be careful with this lane if you need:
-- strict factual preservation on long inputs
-- the strongest style fidelity in the family
-- production-grade rewriting with minimal drift
-In this family, 0.8B is the roughest member. It can do real rewrites, but it is clearly less stable than 2B and 4B.
-## How it was trained
-- Base model: `Qwen/Qwen3.5-0.8B`
-- Training path: Transformers / TRL / PEFT fine-tuning on Hugging Face Jobs
-- Dataset: `N8Programs/unslop-good`
-- Rows used: 1000 (full training split)
-- Objective: direct rewrite / style cleanup
-## Training shape
-- hardware: A10G 24GB (`a10g-large`)
-- max_seq_length: 2048
-- num_train_epochs: 2
-- batch_size: 1
-- gradient_accumulation_steps: 1
-- learning_rate: 1e-4
-- scheduler: cosine
-- warmup_steps: 50
-- LoRA rank: 8
-- LoRA alpha: 20
-- LoRA dropout: 0.0
-- 4-bit loading
-- bf16 training
-## Deployment
-This run is deployment-backed and live on Modal.
-- Status: healthy
 - Base model: `Qwen/Qwen3.5-0.8B`
-- Output repo: `Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1`
-- Mode: standard Unsloth / HF adapter loading
-- Endpoint style: rewrite API
-- Thinking mode: disabled
-Live endpoint generation defaults used by the app:
-- `enable_thinking = false`
-- `temperature = 0.7`
-- `top_p = 0.8`
-- `top_k = 20`
-- `min_p = 0.0`
-- `presence_penalty = 1.5`
-- `repetition_penalty = 1.0`
-## Evaluation snapshot
-Short sanity sample:
-> Rewrite this short-form promo hook to sound less generic and more human.
-Full observed output:
-> The old ad said everything about being the best in town, but that's just a cliché. I'm not trying to be the king of the city. I'm just here to show you how we actually do things. If you want to see the real stuff, come by and let me show you what makes us special. Let's get started.
-Held-out fiction sample (input):
-> The wind rose all at once, as if the mountain itself had inhaled. She stood at the edge of the pass, cloak snapping like a torn banner behind her, staring down at the valley where the last lights were going out one by one. Each flame extinguished felt deliberate—an answer to a question she had been too afraid to ask. Somewhere below, bells began to ring, not in warning, but in mourning. They already knew. “Don’t,” he said, breathless, reaching for her arm. His fingers brushed fabric, then air. “If you cross that line, there’s no coming back.” She didn’t turn. The cold burned her lungs, sharpened her thoughts. “There was never a way back,” she replied softly. “Only a way forward you were willing to pretend didn’t exist.” Thunder rolled, low and ancient. For a moment, the sky fractured into white fire, and in that instant he saw her face—resolved, terrified, incandescent with purpose. Not the girl he had sworn to protect, but the woman the world had demanded she become. “You’ll die,” he said. She smiled then, finally facing him, eyes bright with unshed tears. “So will everyone,” she answered. “The difference is what we leave burning when we’re gone.” She stepped forward. The mountain exhaled.
-## Full observed output on that sample
-> The wind howled, rushing up from the mountainside as if the stone itself had swallowed everything. She sat at the precipice, her cloak fraying like a torn flag behind her, her gaze fixed on the valley where the last fires were flickering out one by one. Each extinguishment felt intentional—a calculated response to a doubt she couldn't articulate yet. Below, bells rang without warning, not as a call to action, but as a mournful chorus. "Don't," the man whispered, his voice thin as smoke, gripping her arm. His skin was rough against her skin; his hands moved across her cloak, then through the air. "Cross that threshold, and you cannot return." She did not look away. The chill hit her bones, making her mind sharper than before. "There was never another way," she murmured, her voice steady. "Only a path you would have chosen to pretend never existed." A thunderstorm rolled in, deep and ancient. For a split second, the sky fractured into clouds of white fire, and in that fleeting moment he caught her face. He saw her resolve, her terror turning into a fierce, burning light. It wasn't the girl he'd sworn to shield; it was the woman the world demanded.
-This endpoint output is no longer a verbatim copy of the input sample; it is now an active rewrite, but with noticeable drift and truncation.
-## Judgment
-Blunt judgment: this is the weakest lane in the family, but it is still a real model, not a toy. It can rewrite long prose in a coherent way, just not as reliably or as naturally as the larger lanes.
-Use it when you care more about cost and latency than final quality.
-## Family position
-- 0.8B: cheapest pilot, roughest output
-- 2B: better balance of fidelity and fluency
-- 4B: strongest default candidate in the small-model set
-- 9B: useful retrain, but still not the safest long-form rewrite choice versus 4B
-## Training loss vs progress
-See the normalized family comparison plot below.
-![Normalized training loss comparison: 9B vs 0.8B vs 2B vs 4B](./training_loss_vs_progress_comparison_9b_0_8b_vs_2b_vs_4b.svg)
-## Bottom line
-If you want the lightest Unslop lane, this is the one to grab. If you want higher rewrite quality, scale up to the larger lanes (9B and especially 30B-A3B for quality-first use).

 - rewriting
 - style-transfer
 - unslop
+pipeline_tag: text-generation
 ---
 # qwen3.5-0.8b-unslop-good-lora-v1
+Unslop rewrite adapter focused on reducing hype/corporate phrasing while preserving meaning.
+## Model summary
+- Repo: `Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1`
 - Base model: `Qwen/Qwen3.5-0.8B`
+- Adapter type: LoRA
+- Pipeline: text generation / rewrite style transfer
+- Current downloads (snapshot): 215
+## Intended use
+- Rewrite AI-sounding drafts into cleaner, more natural prose.
+- Keep meaning and key facts intact.
+- Use as a post-processing layer for longform and social text cleanup.
+## Limitations
+- Can still over-rewrite some passages.
+- Not guaranteed to improve factual accuracy.
+- Should be human-reviewed for fidelity-sensitive outputs.
+## Evaluation notes
+This card records this model as part of the Unslop family with a common quality goal: preserve meaning, reduce hype, and avoid hallucinated additions.
+## Usage (PEFT)
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+base = "Qwen/Qwen3.5-0.8B"
+adapter = "Oysiyl/qwen3.5-0.8b-unslop-good-lora-v1"
+tokenizer = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
+base_model = AutoModelForCausalLM.from_pretrained(base, trust_remote_code=True, device_map="auto")
+model = PeftModel.from_pretrained(base_model, adapter)
+```