Multitask Prompted Training Enables Zero-Shot Task Generalization
Paper • 2110.08207 • Published • 2
How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "crumb/gpt-j-6b-finetune-super-glue" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "crumb/gpt-j-6b-finetune-super-glue",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
YAML Metadata Error:"datasets[0]" with value "The Pile" is not valid. If possible, use a dataset id from https://hf.co/datasets.
side project, not final working good product
see this repo for more information, finetuned with 8-bit Adam on custom transformed super glue tasks, transformed somewhat like the T0 paper
from IPython import display
!pip install transformers==4.14.1 -q
!pip install bitsandbytes-cuda111==0.26.0 -q
!pip install git+https://github.com/aicrumb/transformers-8bit -q
import transformers_8bit
model, tokenizer, config = transformers_8bit.gptj("crumb/gpt-j-6b-finetune-super-glue", device='cuda')
prompt = tokenizer("<QUERY> If birds are in group B, and snakes are in group A, what group are pythons in? <RESPONSE>", return_tensors='pt')
prompt = {key: value.to('cuda') for key, value in prompt.items()}
length = len(prompt['input_ids'][0])
out = model.generate(**prompt, min_length=length+2, max_length=length+2, do_sample=False, pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(out[0]))
"""output
<QUERY> If birds are in group B, and snakes are in group A, what group are pythons in? <RESPONSE> Group A
"""
- Downloads last month
- 3
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "crumb/gpt-j-6b-finetune-super-glue" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "crumb/gpt-j-6b-finetune-super-glue", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'