Instructions to use Devops-hestabit/otherhalf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Devops-hestabit/otherhalf with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Devops-hestabit/otherhalf")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Devops-hestabit/otherhalf")
model = AutoModelForCausalLM.from_pretrained("Devops-hestabit/otherhalf")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Devops-hestabit/otherhalf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Devops-hestabit/otherhalf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Devops-hestabit/otherhalf",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Devops-hestabit/otherhalf

SGLang

How to use Devops-hestabit/otherhalf with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Devops-hestabit/otherhalf" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Devops-hestabit/otherhalf",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Devops-hestabit/otherhalf" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Devops-hestabit/otherhalf",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Devops-hestabit/otherhalf with Docker Model Runner:
```
docker model run hf.co/Devops-hestabit/otherhalf
```

Devops-hestabit commited on May 4, 2023

Commit

9609e77

1 Parent(s): 40a1571

Update README.md

Browse files

Files changed (1) hide show

README.md +2 -53

README.md CHANGED Viewed

@@ -2,58 +2,7 @@
 license: creativeml-openrail-m
 language:
 - en
-thumbnail:
 tags:
 - text generation
-- conversational
-inference: false
----
-# Pygmalion 6B
-## Model description
-Pymalion 6B is a proof-of-concept dialogue model based on EleutherAI's [GPT-J-6B](https://huggingface.co/EleutherAI/gpt-j-6B).
-**Warning:** This model is **NOT** suitable for use by minors. It **will** output X-rated content under certain circumstances.
-## Training data
-The fine-tuning dataset consisted of 56MB of dialogue data gathered from multiple sources, which includes both real _and_ partially machine-generated conversations.
-## Training procedure
-Model weights were initialized from the `uft-6b` ConvoGPT model made available in [this commit](https://huggingface.co/hakurei/convogpt/tree/41b67bfddb6cd97070ffddf708e9720c9cb8d224/6b-uft).
-The model was then further fine-tuned on ~48.5 million tokens for ~5k steps on 4 NVIDIA A40s using DeepSpeed.
-## Intended use
-### The easy way
-We provide a notebook with a Gradio UI for playing around with the model without having to manually format inputs. This notebook can be found [here](https://github.com/PygmalionAI/gradio-ui/blob/master/notebooks/GPU.ipynb).
-### The manual way
-The model can be used as a regular text generation model, but it'll perform best if the input prompt adheres to the following format:
-```
-[CHARACTER]'s Persona: [A few sentences about the character you want the model to play]
-<START>
-[DIALOGUE HISTORY]
-You: [Your input message here]
-[CHARACTER]:
-```
-Where `[CHARACTER]` is, as you can probably guess, the name of the character you want the model to portray, `<START>` should be used verbatim as a delimiter token to separate persona and scenario data from the dialogue, and `[DIALOGUE HISTORY]` is chat history so the model can have some conversational context to draw from. Ideally it'll be pairs of messages like:
-```
-[CHARACTER]: [some dialogue here]
-You: [your response to the dialogue above]
-```
-Apart from chat history, you can also just add example conversations in `[DIALOGUE HISTORY]` to show how the character should speak - ideally at the beginning, so it doesn't get confused as to what's conversation history vs. character definition.
-## Known issues
-We haven't played around with the model enough to enumerate them. Feel free to give us some feedback!

 license: creativeml-openrail-m
 language:
 - en
+thumbnail: null
 tags:
 - text generation
+---