Instructions to use vonjack/Phi-3.5-mini-instruct-hermes-fc-json with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use vonjack/Phi-3.5-mini-instruct-hermes-fc-json with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="vonjack/Phi-3.5-mini-instruct-hermes-fc-json", filename="ggml-model-Q4_K.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use vonjack/Phi-3.5-mini-instruct-hermes-fc-json with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json # Run inference directly in the terminal: llama-cli -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json # Run inference directly in the terminal: llama-cli -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json # Run inference directly in the terminal: ./llama-cli -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json # Run inference directly in the terminal: ./build/bin/llama-cli -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json
Use Docker
docker model run hf.co/vonjack/Phi-3.5-mini-instruct-hermes-fc-json
- LM Studio
- Jan
- Ollama
How to use vonjack/Phi-3.5-mini-instruct-hermes-fc-json with Ollama:
ollama run hf.co/vonjack/Phi-3.5-mini-instruct-hermes-fc-json
- Unsloth Studio new
How to use vonjack/Phi-3.5-mini-instruct-hermes-fc-json with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for vonjack/Phi-3.5-mini-instruct-hermes-fc-json to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for vonjack/Phi-3.5-mini-instruct-hermes-fc-json to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for vonjack/Phi-3.5-mini-instruct-hermes-fc-json to start chatting
- Docker Model Runner
How to use vonjack/Phi-3.5-mini-instruct-hermes-fc-json with Docker Model Runner:
docker model run hf.co/vonjack/Phi-3.5-mini-instruct-hermes-fc-json
- Lemonade
How to use vonjack/Phi-3.5-mini-instruct-hermes-fc-json with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull vonjack/Phi-3.5-mini-instruct-hermes-fc-json
Run and chat with the model
lemonade run user.Phi-3.5-mini-instruct-hermes-fc-json-{{QUANT_TAG}}List all available models
lemonade list
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json# Run inference directly in the terminal:
llama-cli -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-jsonUse pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json# Run inference directly in the terminal:
./llama-cli -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-jsonBuild from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json# Run inference directly in the terminal:
./build/bin/llama-cli -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-jsonUse Docker
docker model run hf.co/vonjack/Phi-3.5-mini-instruct-hermes-fc-jsonModel Summary
Phi-3.5-mini is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available websites - with a focus on very high-quality, reasoning dense data. The model belongs to the Phi-3 model family and supports 128K token context length. The model underwent a rigorous enhancement process, incorporating both supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures.
Hermes Function-Calling V1
This dataset is the compilation of structured output and function calling data used in the Hermes 2 Pro series of models.
This repository contains a structured output dataset with function-calling conversations, json-mode, agentic json-mode and structured extraction samples, designed to train LLM models in performing function calls and returning structured output based on natural language instructions. The dataset features various conversational scenarios where AI agents are required to interpret queries and execute appropriate single or multiple function calls.
The synthetic data generation was led by @interstellarninja in collaboration with @NousResearch, @teknium, @THEODOROS and many others who provided guidance.
Hermes Function Calling Standard
Hermes Function-calling Standard enables creation of LLM agents that are capable of executing API calls directly from user instructions. For instance, when asked to "find a flight from New York to Los Angeles for next Friday," a function-calling agent can interpret the request, generate the necessary function call (e.g., search_flights), and return the results. These agents significantly enhance the utility of AI by enabling direct interactions with APIs, making them invaluable in digital assistants across various domains.
For a complete useage guide of models trained on this data, see our github repo: https://github.com/NousResearch/Hermes-Function-Calling
- Downloads last month
- 17
Model tree for vonjack/Phi-3.5-mini-instruct-hermes-fc-json
Base model
microsoft/Phi-3.5-mini-instruct
Install from brew
# Start a local OpenAI-compatible server with a web UI: llama-server -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json# Run inference directly in the terminal: llama-cli -hf vonjack/Phi-3.5-mini-instruct-hermes-fc-json