Instructions to use dreamgen/opus-v0-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use dreamgen/opus-v0-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="dreamgen/opus-v0-7b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("dreamgen/opus-v0-7b")
model = AutoModelForMultimodalLM.from_pretrained("dreamgen/opus-v0-7b")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use dreamgen/opus-v0-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "dreamgen/opus-v0-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dreamgen/opus-v0-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/dreamgen/opus-v0-7b

SGLang

How to use dreamgen/opus-v0-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "dreamgen/opus-v0-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dreamgen/opus-v0-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "dreamgen/opus-v0-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dreamgen/opus-v0-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use dreamgen/opus-v0-7b with Docker Model Runner:
```
docker model run hf.co/dreamgen/opus-v0-7b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

DreamGen Opus V0 7B

DreamGen Opus is a family of uncensored models fine-tuned for (steerable) story writing and the model also works great for chat / RP. The DreamGen Opus V0 7B model is derived from mistralai/Mistral-7B-v0.1.

WARNING Quantized versions manifest significant performance degradation compared to the original:

Other sizes:

70B: dreamgen/opus-v0-70b

You can try the Opus V0 70B (AWQ) model for free on dreamgen.com.

Prompting

Please see the official documentation for more detailed guide, including how to prompt the model for chat / RP.

The (collaborative / steerable) story writing task teaches the model to respect <setting> and <instruction> inserted into the prompt.

Example prompt:

<setting>
(Setting provides general overview of the story and characters)
This story is a twist on the traditional Little Red Riding Hood story.
In this variation, the Little Red Riding Hood and her grandma are secretely werevoles.
</setting>

(Previous part of the story, potentially empty)

<instruction>
(Setting tells the model what should happen in the next few sentences / paragraphs)
The Little Red Riding hood confronts The Big Bad Wolf, transforming into her wolf form.
</instruction>

Dataset

The fine-tuning dataset consisted of >1M tokens of collaborative writing task examples, each example being up to 4096 tokens. On top of that, >20M tokens of more general, but less instructed examples were included to help preserve generalization.

All prose in the dataset is from actual humans, not AI generated.

Community

Join the DreamGen community on Discord, or follow our X/Twitter account for new model releases and other news. We will soon be releasing models with longer context window, as well as models specifically fine-tuned for character chat & roleplay.

Help us shape the future of DreamGen.

Running the model

The model is should be compatible with any software that supports mistralai/Mistral-7B-v0.1.

Running on DreamGen.com (free)

You can try the model for free at dreamgen.com — note that an account is required.

Running with vLLM

Install vLLM (version at least 0.2.1.post1)
Run python -u -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --model dreamgen/opus-v0-7b
Use any library that's compatible with the OpenAI API to talk to the model

Running with oobabooga/text-generation-webui

Install oobabooga/text-generation-webui
Go to the Model tab
Under Download custom model or LoRA, enter dreamgen/opus-v0-7b
Go to the Text Generation tab
Enter your prompt

License

For personal and academic use: Same license as the base model, in this case Apache 2.0.
For commercial use: Please reach out to hello@dreamgen.com.

Downloads last month: 27

Model tree for dreamgen/opus-v0-7b

Quantizations

5 models

dreamgen
/

opus-v0-7b