Instructions to use DavidAU/MN-Dark-Planet-TITAN-12B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DavidAU/MN-Dark-Planet-TITAN-12B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="DavidAU/MN-Dark-Planet-TITAN-12B-GGUF",
	filename="MN-Dark-Planet-TITAN-12B-D_AU-IQ4_XS.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use DavidAU/MN-Dark-Planet-TITAN-12B-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use DavidAU/MN-Dark-Planet-TITAN-12B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DavidAU/MN-Dark-Planet-TITAN-12B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DavidAU/MN-Dark-Planet-TITAN-12B-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M

Ollama
How to use DavidAU/MN-Dark-Planet-TITAN-12B-GGUF with Ollama:
```
ollama run hf.co/DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M
```

Unsloth Studio

How to use DavidAU/MN-Dark-Planet-TITAN-12B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for DavidAU/MN-Dark-Planet-TITAN-12B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for DavidAU/MN-Dark-Planet-TITAN-12B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for DavidAU/MN-Dark-Planet-TITAN-12B-GGUF to start chatting

Atomic Chat new
Docker Model Runner
How to use DavidAU/MN-Dark-Planet-TITAN-12B-GGUF with Docker Model Runner:
```
docker model run hf.co/DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M
```

Lemonade

How to use DavidAU/MN-Dark-Planet-TITAN-12B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull DavidAU/MN-Dark-Planet-TITAN-12B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.MN-Dark-Planet-TITAN-12B-GGUF-Q4_K_M

List all available models

lemonade list

Discord

by The0Internet - opened Jan 30, 2025

Discussion

The0Internet

Jan 30, 2025

•

edited Jan 31, 2025

I have a RTX 3060 12GB and wanted to know which model of yours would you suggest for conversation and sentence generation, that would fit that specific GPU. You have many models and seem very knowledgeable about all of this. Figured I'd just ask.

DavidAU

Owner Jan 30, 2025

Please clarify what type of sentence / conversation - all the LLMs/AIs can do this , I just need an idea of what "flavor(s)" you are looking for.
Roughly for conversation - "chat" - I would recommended the less "creative models" - or class 1 and class 2, as these are more stable for multi-turn chat.

The0Internet

Jan 31, 2025

•

edited Jan 31, 2025

Sentences and conversations by CEFR level that include the exact {word} at least once. A1, A2, B1, B2, C1, C2.

Sentences would be 20 sentences preceded by the CEFR heading

A1
[sentence]
[sentence]
[sentence]

Conversations would be at most 20 conversations per CEFR level that include the exact {word} at least once. 1 CEFR level per output. A1-A2 would get 2 speakers at 4 total sentences. B1-C2 would get more speakers up to 4.

A1 - A2
[speaker1:]
[speaker2:]
[speaker1:]
[speaker2:]

B1-C2
[speaker1:]
[speaker2:]
[speaker3:]
[speaker4:]
[speaker2:]
[speaker1:]
[speaker3:]

I just wanted to run this at home instead of having to use ChatGPT. I wanted an uncensored version because it would hopefully provide better text generation and should just follow my instructions when ChatGPT would say I can't give you the sentences for pee at A1 because that's not an A1 word. I don't care just do it, you're the computer why are you telling me no. I get that all the AI can do this but I wanted one that is better at creative writing. I don't need a chatbot. I just need to feed it a que of prompts and get the output back in a txt file which I can figure out on my own.

DavidAU

Owner Jan 31, 2025

Hmm ; I would go with Gemma the Writer (and related models). These are not an "uncensored" as some models, however your instruction requirements are unique and Gemma is very good at prose level instruction - that is actually following them.

If I understand your requirements, Gemma models will be a good place to start to hone your instructions to the LLM/AI first.
They maybe use other model(s) that are more uncensored after this...

Otherwise you might be fighting two battles, instead of one to begin with.

The0Internet

Jan 31, 2025

Thanks I'll try that out. Much respect.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment