Image-Text-to-Text
Transformers
Safetensors
multilingual
hunyuan_vl_mot
hunyuan
vision-language
Embodied
image-to-text
2B
end-to-end
MoT
conversational
custom_code
Instructions to use Jayyun6/HY-Embodied-0.5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Jayyun6/HY-Embodied-0.5 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="Jayyun6/HY-Embodied-0.5", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModelForImageTextToText model = AutoModelForImageTextToText.from_pretrained("Jayyun6/HY-Embodied-0.5", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Jayyun6/HY-Embodied-0.5 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Jayyun6/HY-Embodied-0.5" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Jayyun6/HY-Embodied-0.5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Jayyun6/HY-Embodied-0.5
- SGLang
How to use Jayyun6/HY-Embodied-0.5 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Jayyun6/HY-Embodied-0.5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Jayyun6/HY-Embodied-0.5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Jayyun6/HY-Embodied-0.5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Jayyun6/HY-Embodied-0.5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use Jayyun6/HY-Embodied-0.5 with Docker Model Runner:
docker model run hf.co/Jayyun6/HY-Embodied-0.5
| {%- if not add_generation_prompt is defined %} | |
| {%- set add_generation_prompt = false %} | |
| {%- endif %} | |
| {%- set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='', is_first_sp=true, is_first_user=true, is_last_user=false) %} | |
| {%- for message in messages %} | |
| {%- if message['role'] == 'system' %} | |
| {%- if ns.is_first_sp %} | |
| {%- set ns.system_prompt = ns.system_prompt + message['content'] %} | |
| {%- set ns.is_first_sp = false %} | |
| {%- else %} | |
| {% set ns.system_prompt = ns.system_prompt + '\n\n' + message['content'] %} | |
| {%- endif %} | |
| {%- endif %} | |
| {%- endfor %} | |
| {{- bos_token }} | |
| {{- ns.system_prompt }} | |
| {%- if tools %} | |
| {%- if ns.system_prompt != '' %} | |
| {{- '\n\n# Tools\n\nYou may call one or more functions to assist with the user query.' }} | |
| {%- else %} | |
| {{- '# Tools\n\nYou may call one or more functions to assist with the user query.' }} | |
| {%- endif %} | |
| {{- '\n\nYou are provided with function signatures within <tools></tools> XML tags:' }} | |
| {{- '\n<tools>\n' }} | |
| {%- for tool in tools %} | |
| {%- if loop.index0 > 1 %} | |
| {{- '\n' }} | |
| {%- endif %} | |
| {{- tool | tojson }} | |
| {%- endfor %} | |
| {{- '\n</tools>\n\n' }} | |
| {{- 'For function call returns, you should first print <tool_calls>' }} | |
| {{- 'For each function call, you should return object like:\n' }} | |
| {{- '<tool_call>function_name\n```json\nfunction_arguments_in_json_format\n```</tool_call>' }} | |
| {{- 'At the end of function call returns, you should print </tool_calls>' }} | |
| {%- endif %} | |
| {%- if ns.system_prompt != '' or tools %} | |
| {{- '<|hy_place▁holder▁no▁3|>' }} | |
| {%- endif %} | |
| {%- set image_count = namespace(value=0) %} | |
| {%- set video_count = namespace(value=0) %} | |
| {%- set last_user_idx = namespace(value=-1) %} | |
| {%- for message in messages %} | |
| {%- if message['role'] == 'user' %} | |
| {%- set last_user_idx.value = loop.index0 %} | |
| {%- endif %} | |
| {%- endfor %} | |
| {%- for message in messages %} | |
| {%- if message['role'] == 'user' %} | |
| {%- set ns.is_tool = false %} | |
| {%- set ns.is_first = false %} | |
| {%- set ns.is_last_user = true %} | |
| {{- '<|hy_User|>'}} | |
| {%- if message.content is string %} | |
| {{- message.content }} | |
| {%- else %} | |
| {%- for content in message.content %} | |
| {%- if content.type == 'image' or 'image' in content or 'image_url' in content %} | |
| {%- set image_count.value = image_count.value + 1 %} | |
| {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%} | |
| <|hy_place▁holder▁no▁666|><|hy_place▁holder▁no▁669|><|hy_place▁holder▁no▁672|><|hy_place▁holder▁no▁667|> | |
| {%- elif content.type == 'video' or 'video' in content %} | |
| {%- set video_count.value = video_count.value + 1 %} | |
| {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%} | |
| <|hy_place▁holder▁no▁666|><|hy_place▁holder▁no▁670|><|hy_place▁holder▁no▁672|><|hy_place▁holder▁no▁667|> | |
| {%- elif 'text' in content %} | |
| {{- content.text }} | |
| {%- endif %} | |
| {%- endfor %} | |
| {%- endif %} | |
| {%- if loop.index0 == last_user_idx.value %} | |
| {%- if enable_thinking is defined and enable_thinking %} | |
| {{- '/think' }} | |
| {%- else %} | |
| {{- '/no_think' }} | |
| {%- endif %} | |
| {%- else %} | |
| {{- '/no_think' }} | |
| {%- endif %} | |
| {%- endif %} | |
| {%- if message['role'] == 'assistant' and message['tool_calls'] is defined and message['tool_calls'] is not none %} | |
| {%- set ns.is_last_user = false %} | |
| {%- if ns.is_tool %} | |
| {{- '</tool_responses>' }} | |
| {%- endif %} | |
| {{- '<|hy_Assistant|>' }} | |
| {%- set ns.is_first = false %} | |
| {%- set ns.is_tool = false %} | |
| {%- set ns.is_output_first = true %} | |
| {%- for tool in message['tool_calls'] %} | |
| {%- set arguments = tool['function']['arguments'] %} | |
| {%- if arguments is not string %} | |
| {%- set arguments = arguments | tojson %} | |
| {%- endif %} | |
| {%- if not ns.is_first %} | |
| {%- if message['content'] is none %} | |
| {{- '<tool_calls><tool_call>' + tool['function']['name'] + '\n```json\n' + arguments + '\n```</tool_call>' }} | |
| {%- else %} | |
| {{- message['content'] + '<tool_calls><tool_call>' + tool['function']['name'] + '\n```json\n' + arguments + '\n```</tool_call>' }} | |
| {%- endif %} | |
| {%- set ns.is_first = true %} | |
| {%- else %} | |
| {{- '\n<tool_call>' + tool['function']['name'] + '\n```json\n' + arguments + '\n```</tool_call>' }} | |
| {%- endif %} | |
| {%- endfor %} | |
| {{- '</tool_calls>' + eos_token }} | |
| {%- endif %} | |
| {%- if message['role'] == 'assistant' and (message['tool_calls'] is not defined or message['tool_calls'] is none) %} | |
| {%- set ns.is_last_user = false %} | |
| {{- '<|hy_Assistant|>' }} | |
| {{- '<think>\n\n</think>\n' }} | |
| {{- '<answer>\n' }} | |
| {%- if message['content'] is string %} | |
| {{- message['content'] }} | |
| {%- else %} | |
| {%- for content_item in message['content'] %} | |
| {%- if 'text' in content_item %} | |
| {{- content_item['text'] }} | |
| {%- endif %} | |
| {%- endfor %} | |
| {%- endif %} | |
| {{- '\n</answer>' }} | |
| {{- eos_token }} | |
| {%- endif %} | |
| {%- if message['role'] == 'tool' %} | |
| {%- set ns.is_last_user = false %} | |
| {%- set ns.is_tool = true %} | |
| {%- if ns.is_output_first %} | |
| {{- '<|hy_User|>' + '<tool_responses><tool_response>' + message['content'] + '</tool_response>' }} | |
| {%- set ns.is_output_first = false %} | |
| {%- else %} | |
| {{- '\n<tool_response>' + message['content'] + '</tool_response>' }} | |
| {%- endif %} | |
| {%- endif %} | |
| {%- endfor %} | |
| {%- if ns.is_tool %} | |
| {{- '</tool_responses>' }} | |
| {%- endif %} | |
| {%- if add_generation_prompt %} | |
| {{- '<|hy_Assistant|>' }} | |
| {%- if enable_thinking is defined and enable_thinking %} | |
| {{- '<think>' }} | |
| {%- else %} | |
| {{- '<think>\n\n</think>\n' }} | |
| {%- endif %} | |
| {%- endif %} |