Instructions to use nothingiisreal/MN-12B-Celeste-V1.9 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nothingiisreal/MN-12B-Celeste-V1.9 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nothingiisreal/MN-12B-Celeste-V1.9")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("nothingiisreal/MN-12B-Celeste-V1.9")
model = AutoModelForMultimodalLM.from_pretrained("nothingiisreal/MN-12B-Celeste-V1.9")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Local Apps Settings

vLLM

How to use nothingiisreal/MN-12B-Celeste-V1.9 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nothingiisreal/MN-12B-Celeste-V1.9"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nothingiisreal/MN-12B-Celeste-V1.9",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nothingiisreal/MN-12B-Celeste-V1.9

SGLang

How to use nothingiisreal/MN-12B-Celeste-V1.9 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nothingiisreal/MN-12B-Celeste-V1.9" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nothingiisreal/MN-12B-Celeste-V1.9",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nothingiisreal/MN-12B-Celeste-V1.9" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nothingiisreal/MN-12B-Celeste-V1.9",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nothingiisreal/MN-12B-Celeste-V1.9 with Docker Model Runner:
```
docker model run hf.co/nothingiisreal/MN-12B-Celeste-V1.9
```

Feedback

by Darkknight535 - opened Aug 13, 2024

Discussion

Darkknight535

Aug 13, 2024

the model is overall the best, it's creative and the best in roleplaying, just one issue
when i used some prompts to output an image generation prompt, it does not provide output as i want, like in comma delimited keywords, it continues roleplaying which is an issue. though the base nemo instruct follows things well.

Ardvark123

Aug 13, 2024

I was up all night on this thing its really amazing. The best part no gptisms I was so happy.

Darkknight535

Aug 14, 2024

Will anyone help me??

Altotas

Aug 14, 2024

Will anyone help me??

Have you tried Starcannon v2 or v3? They might be better at this while still keeping Celeste's quality.

Darkknight535

Aug 14, 2024

okay

odyss3y

Aug 15, 2024

try using OOC: (i think the default is just wrapping the request in square brackets?) and tell it to use visually aesthetic CLIP terms in comma separated format. if it has trouble still staying in character you can mention in the system prompt early that it will receive ooc requests in the future and to respond accordingly before proceeding with the story

Darkknight535

Aug 16, 2024

i've tried doing it still no fix, though i made a script in st that changes the model for a while for summary and image generation.

Darkknight535

Aug 16, 2024

// save current values |
/api | /setvar key=saveApi || /preset | /setvar key=savePreset || /context | /setvar key=saveContext ||
// set your preset, model, api here |
/api cohere ||
// get SD freemode prompt and generate |
/summarize ||
/sd you ||
// restore previous values |
/api {{getvar::saveApi}} ||
/flushvar saveApi || /flushvar savePreset ||

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment