Instructions to use OpenAssistant/pythia-12b-sft-v8-7k-steps with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenAssistant/pythia-12b-sft-v8-7k-steps with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="OpenAssistant/pythia-12b-sft-v8-7k-steps")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("OpenAssistant/pythia-12b-sft-v8-7k-steps") model = AutoModelForCausalLM.from_pretrained("OpenAssistant/pythia-12b-sft-v8-7k-steps") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use OpenAssistant/pythia-12b-sft-v8-7k-steps with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OpenAssistant/pythia-12b-sft-v8-7k-steps" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenAssistant/pythia-12b-sft-v8-7k-steps", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/OpenAssistant/pythia-12b-sft-v8-7k-steps
- SGLang
How to use OpenAssistant/pythia-12b-sft-v8-7k-steps with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "OpenAssistant/pythia-12b-sft-v8-7k-steps" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenAssistant/pythia-12b-sft-v8-7k-steps", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "OpenAssistant/pythia-12b-sft-v8-7k-steps" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenAssistant/pythia-12b-sft-v8-7k-steps", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use OpenAssistant/pythia-12b-sft-v8-7k-steps with Docker Model Runner:
docker model run hf.co/OpenAssistant/pythia-12b-sft-v8-7k-steps
Provide a context
Hi all,
I have the following question: Is it possible to provide a context to the model?
I mean, providing a file or a list of chat to the model is it possible to narrow the model response to the that file/chat?
I believe the context is added inside the prompt surrounded by the correct tokens. For example:
<|prompter|>List 5 crops<|endoftext|><|assistant|><|endoftext|>1. Corn
2. Rice
3. Wheat
4. Soybeans
5. Maize (Corn)
(Note: Some people may argue that soybeans and corn are the same thing, but I've included them as separate items for clarity.)<|endoftext|><|prompter|>tell me more about number 2<|endoftext|><|assistant|>Rice is a staple food in many parts of Asia, Africa and South America...<|endoftext|>
If you wanted to include context from another text file or something, I think you would surround that with the <|prefix_begin|> and <|prefix_end|> tokens. This is what I've been using and it's worked well so far. Here's another example:
<|system|>You are an AI that summarizes a conversation in as few words as possible.<|endoftext|><|prefix_begin|>Rice is a staple food in many parts of Asia, Africa and South America. It
has been an important part of human diets for thousands of years. The cultivation of rice was first recorded in ancient China over 3,000 years ago, where it quickly spread to other countries around the world. Today, there are many different types of rice, each with its own unique flavor and cooking properties.
The most common type of rice used today is called long-grain rice, which has a tender texture when cooked and a slightly nutty aroma. Short- or medium-grained rice is better suited for dishes like paella or biryani, while long- or short-grip can be enjoyed as a savory side dish such as fried rice or pilaf.
If you're looking to try some delicious recipes using rice check out these sites below!
http://www.seriouseats.com/recipes/2016/05/best-rice-dish-recipes-youll-love.html
http:/blog.mexico-net.com/?p=3857
I hope this helps. Have fun cooking!
Don't forget to subscribe and like us on social media for daily updates. β
β
β
β
peace. βπΌβ€οΈ.<|prefix_end|><|prompter|>Summarize the previous conversation in as few words as possible<|endoftext|><|
assistant|>"Rice is a common grain found in various cuisines worldwide." "There exist multiple types of rices including long grain, medium grain, etcetera." There are numerous recipes one could make from rice depending on their preferences." <|endoftext|>