Instructions to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF", dtype="auto")

llama-cpp-python

How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF",
	filename="Mecellem-Qwen3-1.7B-TR.i1-IQ1_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

SGLang

How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with Ollama:
```
ollama run hf.co/mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M
```

Unsloth Studio

How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF to start chatting

How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/Mecellem-Qwen3-1.7B-TR-i1-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Mecellem-Qwen3-1.7B-TR-i1-GGUF-Q4_K_M

List all available models

lemonade list

mradermacher commited on Jan 23

Commit

f13861f

verified ·

1 Parent(s): 10aea02

uploaded from rain

Browse files

Files changed (25) hide show

.gitattributes +24 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ1_M.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ1_S.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ2_M.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ2_S.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ2_XS.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ2_XXS.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ3_M.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ3_S.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ3_XS.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ3_XXS.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ4_NL.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-IQ4_XS.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q2_K.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q2_K_S.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q3_K_L.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q3_K_M.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q3_K_S.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q4_0.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q4_1.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q4_K_M.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q4_K_S.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q5_K_M.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q5_K_S.gguf +3 -0
Mecellem-Qwen3-1.7B-TR.i1-Q6_K.gguf +3 -0

@@ -34,3 +34,27 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 Mecellem-Qwen3-1.7B-TR.imatrix.gguf filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 Mecellem-Qwen3-1.7B-TR.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Mecellem-Qwen3-1.7B-TR.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

+version https://git-lfs.github.com/spec/v1
+oid sha256:cc797a117960b8a4a9fcee8eb707f1b21398d6e82afef360c10691e4d5b2c7ee
+size 645895200

+version https://git-lfs.github.com/spec/v1
+oid sha256:b07188955d62b49dcc79b5ec3c387e43067ec7bd0098214dc5b5ced69f090df9
+size 617878560

+version https://git-lfs.github.com/spec/v1
+oid sha256:ad52878637b94a614755733fe600f476c22ffafdd20b94c59a0d0704c7692847
+size 828886048

+version https://git-lfs.github.com/spec/v1
+oid sha256:e7d659a9a0eb0b1ad4f50c061a15a5d3788c0dbf6bfba344ff07b9e13a399c41
+size 791530528

+version https://git-lfs.github.com/spec/v1
+oid sha256:3800252bc12e2f9afa8d8119faa93a774675dffd8ba2dc64047954d7b1f26b40
+size 733615136

+version https://git-lfs.github.com/spec/v1
+oid sha256:15cf4dfca3fd2e6c7ac6fff1b1a6f1d2f13e36be665ab0566f5f815393054e05
+size 692589600

+version https://git-lfs.github.com/spec/v1
+oid sha256:7d889b447e61537f0942baea10a87e3ad9dc2b36f31a5a16684fed0f3b995c8b
+size 1029366816

+version https://git-lfs.github.com/spec/v1
+oid sha256:396f6e83bc2a5818e5222575f153f0bbff972ce36be53458f4de4e1a64daa893
+size 1000956960

+version https://git-lfs.github.com/spec/v1
+oid sha256:01ec36c8ef1eb3a704c1a9c9d9bc27b64d2b38668c75fe325f984ae99256f523
+size 967926816

+version https://git-lfs.github.com/spec/v1
+oid sha256:be1e2d350c772f6175765a3b04218db15301461aeba63dd9808fef798a280fed
+size 888065056

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:86294d0d0734b6f9ff35c874ec1ae439c75a156426be1664ca576e1bf8de4d8b
+size 1229454368

+version https://git-lfs.github.com/spec/v1
+oid sha256:980f100947c9dc425880039a8cda233b52a3899f50d5225f3a588594f52e26ae
+size 1175690272

+version https://git-lfs.github.com/spec/v1
+oid sha256:3395dbce5247e61f4918ac5c6000606eeda7706f2e7ea83baee895247650ab50
+size 879897632

+version https://git-lfs.github.com/spec/v1
+oid sha256:e6fa1f3fe75f50ae7d4aa926928e72d6399687f33773e51043791e68d62f4dbd
+size 835071008

+version https://git-lfs.github.com/spec/v1
+oid sha256:057d45dc5a721a08602e2d67d1cb023010d981405bf04088d7f026f44973f549
+size 1137206304

+version https://git-lfs.github.com/spec/v1
+oid sha256:d9e24b291f9e8870a38fe6448965a3d0f6c289f955841e6300d6cf403afa0621
+size 1073243168

+version https://git-lfs.github.com/spec/v1
+oid sha256:567831d1718a04b01c141e4e9eeaf6d8d720366f1ffbcf274f436f4593236e6d
+size 1231813664

+version https://git-lfs.github.com/spec/v1
+oid sha256:e4dc4fe958900925f47c06453e49aff21996027c57ce8cd1c9834eb9f1cc4029
+size 1336982560

+version https://git-lfs.github.com/spec/v1
+oid sha256:dbd1e40c7e658d626ff62c93fdba66a6aa1bd12820ff62665a7d526b79e2c35a
+size 1282440224

+version https://git-lfs.github.com/spec/v1
+oid sha256:df3016e7a70cdb85af335f4d753c4faef9ae7fe51d2cb13f5ff1d742bd8a511f
+size 1235221536

+version https://git-lfs.github.com/spec/v1
+oid sha256:b1d43b69550035c8e59748b311c67cd1ecbce2b02bef7c9e3bb437efc9f54689
+size 1471806496

+version https://git-lfs.github.com/spec/v1
+oid sha256:50685a35b7c4ec015fde0ca50a851acef84b44821096a2efe2ccd93daa0c07f3
+size 1444510752

+version https://git-lfs.github.com/spec/v1
+oid sha256:8f92c084e4f922df79cc009f6bfb39a5d9fbea1b91083e822f3bcc2e23fa2566
+size 1673008160