Instructions to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers
# Run inference directly in the terminal:
llama cli -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers
# Run inference directly in the terminal:
llama cli -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers
# Run inference directly in the terminal:
./llama-cli -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers
# Run inference directly in the terminal:
./build/bin/llama-cli -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Use Docker

docker model run hf.co/meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

LM Studio
Jan

vLLM

How to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "meshllm/gpt-oss-20b-UD-Q4_K_XL-layers"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meshllm/gpt-oss-20b-UD-Q4_K_XL-layers",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Ollama
How to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with Ollama:
```
ollama run hf.co/meshllm/gpt-oss-20b-UD-Q4_K_XL-layers
```

Unsloth Studio

How to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for meshllm/gpt-oss-20b-UD-Q4_K_XL-layers to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for meshllm/gpt-oss-20b-UD-Q4_K_XL-layers to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for meshllm/gpt-oss-20b-UD-Q4_K_XL-layers to start chatting

How to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "meshllm/gpt-oss-20b-UD-Q4_K_XL-layers"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "meshllm/gpt-oss-20b-UD-Q4_K_XL-layers" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with Docker Model Runner:
```
docker model run hf.co/meshllm/gpt-oss-20b-UD-Q4_K_XL-layers
```

Lemonade

How to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Run and chat with the model

lemonade run user.gpt-oss-20b-UD-Q4_K_XL-layers-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use meshllm/gpt-oss-20b-UD-Q4_K_XL-layers with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default meshllm/gpt-oss-20b-UD-Q4_K_XL-layers

Run Hermes

hermes

Atomic Chat

gpt-oss-20b-UD-Q4_K_XL-layers / model-package.json

jamesdumay

Add layer package manifest from unsloth/gpt-oss-20b-GGUF (unsloth/gpt-oss-20b-GGUF:UD-Q4_K_XL)

d6547bf verified 2 months ago

Raw

History Blame Contribute Delete

7.6 kB

	{
	"schema_version": 1,
	"model_id": "unsloth/gpt-oss-20b-GGUF:UD-Q4_K_XL",
	"source_model": {
	"path": "/source/gpt-oss-20b-UD-Q4_K_XL.gguf",
	"sha256": "10fe673de12c20b74b8d670a9fdf0fd36b43b0a86ffc04daeb175c0a2b98c4f9",
	"repo": "unsloth/gpt-oss-20b-GGUF",
	"revision": "main",
	"primary_file": "gpt-oss-20b-UD-Q4_K_XL.gguf",
	"canonical_ref": "unsloth/gpt-oss-20b-GGUF@main/gpt-oss-20b-UD-Q4_K_XL.gguf",
	"distribution_id": "gpt-oss-20b-UD-Q4_K_XL",
	"files": [
	{
	"path": "gpt-oss-20b-UD-Q4_K_XL.gguf",
	"size_bytes": null,
	"sha256": null
	}
	]
	},
	"format": "layer-package",
	"layer_count": 24,
	"activation_width": 2880,
	"shared": {
	"metadata": {
	"path": "shared/metadata.gguf",
	"tensor_count": 0,
	"tensor_bytes": 0,
	"artifact_bytes": 12981184,
	"sha256": "480835ff41f925215ad8131164b87bac6a0e961ea0864980a21bddc4550d0454"
	},
	"embeddings": {
	"path": "shared/embeddings.gguf",
	"tensor_count": 1,
	"tensor_bytes": 615329280,
	"artifact_bytes": 628310496,
	"sha256": "30d41b9c4d3827cdf4dd0779641db2689989b5052bf329f704f3ee4d884b7f86"
	},
	"output": {
	"path": "shared/output.gguf",
	"tensor_count": 2,
	"tensor_bytes": 615340800,
	"artifact_bytes": 628322080,
	"sha256": "ddbd3aabab3f8fb37ffdb2c1bf77d4a8d380c1ff57719c05aa52edcc20983e7b"
	}
	},
	"layers": [
	{
	"layer_index": 0,
	"path": "layers/layer-000.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441870464,
	"artifact_bytes": 454852768,
	"sha256": "b3858aa5b0ea3776e0df37073f2c875d209fa3bd167a2c30bf6997d34330f4a8"
	},
	{
	"layer_index": 1,
	"path": "layers/layer-001.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441870464,
	"artifact_bytes": 454852768,
	"sha256": "a68eaec12463629e23ff3d55eee49487aa72debb8aa980fb11356ad6c66d4680"
	},
	{
	"layer_index": 2,
	"path": "layers/layer-002.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441870464,
	"artifact_bytes": 454852768,
	"sha256": "bfc367f2c090c35527a4c518bc6467e052fd4720a2eccf5885f5bb9a925566d2"
	},
	{
	"layer_index": 3,
	"path": "layers/layer-003.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441317504,
	"artifact_bytes": 454299808,
	"sha256": "625b2a7838c8d2214319bdb0cfd49bf5f4be73c04ed98ef9ef04800b5ad9c6db"
	},
	{
	"layer_index": 4,
	"path": "layers/layer-004.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441317504,
	"artifact_bytes": 454299808,
	"sha256": "bf89ff2e5c83dd3ca1e069181107b8ef77b88949bc87dbcbbcdf6a1b29582a61"
	},
	{
	"layer_index": 5,
	"path": "layers/layer-005.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444911744,
	"artifact_bytes": 457894048,
	"sha256": "462e5700b0f8b2f9e6a5a226b6eb01e7c1f1b0b9027cd4bf4c2d3c9a0ef233cd"
	},
	{
	"layer_index": 6,
	"path": "layers/layer-006.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441317504,
	"artifact_bytes": 454299808,
	"sha256": "e095ce622785959af3fa82c30d242603d995f2bdca75797e332733e111ad5ccb"
	},
	{
	"layer_index": 7,
	"path": "layers/layer-007.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441317504,
	"artifact_bytes": 454299808,
	"sha256": "88b99aeb18d63060c5a61a4ed53a7e8e7985a5d100ce19e08321549e78774ee8"
	},
	{
	"layer_index": 8,
	"path": "layers/layer-008.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441870464,
	"artifact_bytes": 454852768,
	"sha256": "869e24bded2d736d9a4ea18c1e85b7d50cb26459244c6f13cfe4f8553e75ebac"
	},
	{
	"layer_index": 9,
	"path": "layers/layer-009.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444358784,
	"artifact_bytes": 457341088,
	"sha256": "50f256ebd32dd61eb53ecc3e0d7c520cef0a1dc8acd615afedf39c87edff1734"
	},
	{
	"layer_index": 10,
	"path": "layers/layer-010.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444358784,
	"artifact_bytes": 457341120,
	"sha256": "c080b5e9e45b3917a01aa3e1ec799d702431b84965d653ef8b5b71a6aeaf860a"
	},
	{
	"layer_index": 11,
	"path": "layers/layer-011.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441870464,
	"artifact_bytes": 454852800,
	"sha256": "568f9e120925f797f856ed9ce2d8c3bc5cf57e167934af8744b26c383f6d1dff"
	},
	{
	"layer_index": 12,
	"path": "layers/layer-012.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444358784,
	"artifact_bytes": 457341120,
	"sha256": "5f8ed8d2ea5aa15833b8111e24e33650d354468e7af1e97a392992a8b218e2d9"
	},
	{
	"layer_index": 13,
	"path": "layers/layer-013.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444358784,
	"artifact_bytes": 457341120,
	"sha256": "72c00c9eb7e058e4255264acd5c7b646a9feab04644c229c0b7eb22545d585d2"
	},
	{
	"layer_index": 14,
	"path": "layers/layer-014.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441870464,
	"artifact_bytes": 454852800,
	"sha256": "4efc097b88d92be235e94843866c203a622ee09ab84cb70ae81969ab2a5321e4"
	},
	{
	"layer_index": 15,
	"path": "layers/layer-015.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441317504,
	"artifact_bytes": 454299840,
	"sha256": "6bd9aefe233ba8c68191ce6ca8d8971f62abb4a37bccf09f11c738b2255df911"
	},
	{
	"layer_index": 16,
	"path": "layers/layer-016.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441317504,
	"artifact_bytes": 454299840,
	"sha256": "667afc134908bcbed327831baea9e56b38ec07de78e4121fd36d4067dc6a4eb0"
	},
	{
	"layer_index": 17,
	"path": "layers/layer-017.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441870464,
	"artifact_bytes": 454852800,
	"sha256": "6b517a5ad668a47dbc9bf29241736c809431c2aa60751799ee9bd0f1c0d36ddb"
	},
	{
	"layer_index": 18,
	"path": "layers/layer-018.gguf",
	"tensor_count": 19,
	"tensor_bytes": 441317504,
	"artifact_bytes": 454299840,
	"sha256": "beb7bf585bab2dd0452532fbf2afa7225535e25786eca635f1c43c4a9ce1d92b"
	},
	{
	"layer_index": 19,
	"path": "layers/layer-019.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444358784,
	"artifact_bytes": 457341120,
	"sha256": "4be17042456dd8b06488276b74e9c72d8bff423cf878ea73c5b030c338ced035"
	},
	{
	"layer_index": 20,
	"path": "layers/layer-020.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444911744,
	"artifact_bytes": 457894080,
	"sha256": "e5077a64a82680a9c7e22c38d6f8a3da2d42cd824406d3cb3067cd5e44d56de3"
	},
	{
	"layer_index": 21,
	"path": "layers/layer-021.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444911744,
	"artifact_bytes": 457894080,
	"sha256": "3cd5bac00cee87265fbc5579b5308822e55880cb5c24eab930d704820ba8687a"
	},
	{
	"layer_index": 22,
	"path": "layers/layer-022.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444911744,
	"artifact_bytes": 457894080,
	"sha256": "6c81c1b9ea9615d1cec5e7bf87a4eb877cd365e78eca41c1170a2f97e8904170"
	},
	{
	"layer_index": 23,
	"path": "layers/layer-023.gguf",
	"tensor_count": 19,
	"tensor_bytes": 444911744,
	"artifact_bytes": 457894080,
	"sha256": "2264a13efc5d277048c8b913cecbc8f6e8383b4430912764889e790b10d88541"
	}
	],
	"skippy_abi_version": "0.1.25",
	"created_at_unix_secs": 1780348607
	}