Base Model : manojpreveen/mpt-30b-v4
Tool : MosaicML's llm-foundry (https://github.com/mosaicml/llm-foundry)
Dataset : Entire flan1m-GPT4 dataset
Config yaml with Model Params : https://huggingface.co/manojpreveen/mpt-30b-v5/blob/main/mpt-30b_v5.yaml
Description : mosaicml/mpt-30b -> Finetuning on (Entire flan3m-GPT3.5 dataset for 4 epochs) iamplus/mpt-30b-v4 -> Finetuning on (Entire flan1m-GPT4 dataset for 4 epochs) -> iamplus/mpt-30b-v5
Prompt Format :
<system>: [system prompt]
<human>: [question]
<bot>:
- Downloads last month
- 9
Install from pip and serve model
# Install vLLM from pip: pip install vllm# Start the vLLM server: vllm serve "iamplus/mpt-30b-v5"# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "iamplus/mpt-30b-v5", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'