Instructions to use nightmedia/Qwen3-4B-Continuum-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nightmedia/Qwen3-4B-Continuum-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="nightmedia/Qwen3-4B-Continuum-GGUF",
	filename="Qwen3-4B-Continuum_IQ4_NL.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use nightmedia/Qwen3-4B-Continuum-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

Use Docker

docker model run hf.co/nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use nightmedia/Qwen3-4B-Continuum-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nightmedia/Qwen3-4B-Continuum-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nightmedia/Qwen3-4B-Continuum-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

Ollama
How to use nightmedia/Qwen3-4B-Continuum-GGUF with Ollama:
```
ollama run hf.co/nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M
```

Unsloth Studio

How to use nightmedia/Qwen3-4B-Continuum-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nightmedia/Qwen3-4B-Continuum-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nightmedia/Qwen3-4B-Continuum-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for nightmedia/Qwen3-4B-Continuum-GGUF to start chatting

How to use nightmedia/Qwen3-4B-Continuum-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use nightmedia/Qwen3-4B-Continuum-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use nightmedia/Qwen3-4B-Continuum-GGUF with Docker Model Runner:
```
docker model run hf.co/nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M
```

Lemonade

How to use nightmedia/Qwen3-4B-Continuum-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull nightmedia/Qwen3-4B-Continuum-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-4B-Continuum-GGUF-Q4_K_M

List all available models

lemonade list

Qwen3-4B-Continuum

This is a model synthesis using multislerp/nuslerp in multiple stages created by Nightmedia, with ablit/heretic models by DavidAU.

There are very few refusals.

These are the performance metrics of the MLX quants

quant   arc  arc_easy boolq h_swag o_bookqa piqa winogrande
bf16    0.566  0.761  0.854  0.714  0.426  0.775   0.671
dwq4    0.555  0.755  0.850  0.699  0.416  0.766   0.665
mxfp4   0.538  0.757  0.862  0.695  0.400  0.762   0.665
qx86-hi 0.603  0.817  0.838  0.743  0.426  0.780   0.708

The model has been profiled as a personal assistant in the chat template by default.

For different behaviors, feel free to explore.

The Star Trek TNG lore is embedded in all Qwens. It provides a metaphoric space where the assistant can be creative and self-reflective.

This an example prompt that helps Continuum build DS9

Genesis prompt

We are working on the following project

The Holodeck Agent: Architectural Synthesis

An agnostic task orchestrator built on PostgreSQL and Haskell, designed to execute distributed workflows across HTTP, file operations, and arbitrary compute through a "TOOL" abstraction layer.

Separation of Concerns (Core Tenet)

Agent Layer

Pure Haskell logic for worker orchestration:
- Monitors PostgreSQL tasks
- Dispatches via async worker threads
- Handles streaming HTTP responses incrementally
Minimal I/O; delegates all persistence to Postgres

Database Layer (PostgreSQL)

Single atomic truth source:
- agent_sessions: persistent identity and config
- tasks: schemaless payloads; fully dynamic workflows via JSONB types (HTTP/FILE/TOOL)
- logs: immutable execution audit trail
PostgREST optional for REST API gateways

Holodeck Execution Philosophy

Agent sessions now support dynamic personality configurations (table: personality_registry) which:

Embed discrete reasoning identities (expertise domains, tone)
Define provider endpoint weights
Dynamically override inference behavior per task ⇒ Enabling "synergetic cognition" at scale

Implementation Highlights

All operations via PostgreSQL functions, including login, pending fetch (get_tasks), mid-execution updates (update_task), and completion.
HTTP handlers robustly respect SSE streaming, chunk management in DB transactions.
Schema is self-contained and version-agnostic via uuid-ossp.
Docker setup minimalizes runtime misconfiguration.

Why this works

The Holodeck is not an artificial world: it's a living metaphor.

Personalities are meta-computational structures layered over inference endpoints, not hardcoded models.
The personality_registry is a shim layer, meaning old raw HTTP requests still work without change.

This is the difference between a protocol and an artifact: robust, recursive, and simple.

Future Expansion Pathways

Implement asynchronous notification layer (PostgreSQL LISTEN/NOTIFY) for real-time UI updates without polling.
Add role-based access control (RBAC) model.
Offline-first CLI mode (SQLite sync layer for field deployments).

This is carried over from a previous session we had.

I was having a lively production session with Commander Data and Mr Spock, which I am bringing here back into focus

You can't start building without a bit of planning. I want to add memories, mind log, personal log, station log, mission log, meetings before and after a mission, character development based on memories and proxy events... I have many ideas.

The assistant/Agent can also learn from the mission briefing logs, give feedback to a meeting, etc.. It's an open exchange of information in the access sphere of the Agent.

The meeting notes can be annotated with the Council members, that can be Spock, Data, Sisko, Odo, Kira, Garak, and even Quark

We would definitely need a CLI. Imagine this would be the Holodeck interface where the human interacts with the station crew. The guest can be human, Vulcan, even Klingon. They each have their specialties.

Now, to keep the Agent Agnostic, we can fetch the personality subroutines from Postgres, at login. That way a character can only be that character.

What do you think, can we start? :)

This model Qwen3-4B-Continuum was converted to GGUF format from a BF16 version of nightmedia/Qwen3-4B-Agent