Instructions to use pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf",
	filename="Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge.q2_k.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M

Use Docker

docker model run hf.co/pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M

LM Studio
Jan
Ollama
How to use pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf with Ollama:
```
ollama run hf.co/pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M
```

Unsloth Studio

How to use pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf to start chatting

How to use pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf with Docker Model Runner:
```
docker model run hf.co/pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M
```

Lemonade

How to use pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull pravdin/Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf:Q4_K_M

Run and chat with the model

lemonade run user.Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge-gguf-Q4_K_M

List all available models

lemonade list

Quantized GGUF model Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge

This model has been quantized using llama-quantize from llama.cpp

Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge

Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge is a merge of the following models using mergekit:

🧩 Merge Configuration

merge_method: linear
base_model: mistralai/Mistral-7B-Instruct-v0.3
models:
  - model: NousResearch/Hermes-2-Pro-Mistral-7B
    parameters:
      weight: 0.3
  - model: mistralai/Mistral-7B-Instruct-v0.3
    parameters:
      weight: 0.7
parameters:
  normalize: true
dtype: float16

Model Description

The Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge combines the advanced conversational capabilities of the Hermes 2 Pro model with the instruction-following prowess of the Mistral-7B-Instruct model. This strategic fusion aims to enhance the model's ability to understand and generate contextually relevant responses while maintaining a high level of performance across various natural language processing tasks.

Hermes 2 Pro is an upgraded version of the original Nous Hermes 2, featuring a refined dataset and improved function calling capabilities. It excels in generating structured outputs, making it particularly useful for applications requiring precise data formatting, such as JSON responses. The Mistral-7B-Instruct model, on the other hand, is designed to follow instructions effectively, making it a strong candidate for tasks that require adherence to user prompts.

Use Cases

This merged model is well-suited for a variety of applications, including but not limited to:

Conversational agents and chatbots
Function calling and structured data generation
Instruction-based tasks and question answering
Creative writing and storytelling

Model Features

Enhanced Conversational Abilities: The model leverages the conversational strengths of Hermes 2 Pro, allowing for engaging and context-aware dialogues.
Instruction Following: With the integration of Mistral-7B-Instruct, the model can effectively follow user instructions, making it ideal for task-oriented applications.
Function Calling and JSON Outputs: The model supports advanced function calling and can generate structured JSON outputs, facilitating integration with various applications and APIs.

Evaluation Results

The performance of the parent models provides a solid foundation for the merged model. Here are some evaluation metrics from the original models:

Hermes 2 Pro

Function Calling Accuracy: 91%
JSON Mode Accuracy: 84%

Mistral-7B-Instruct

While specific evaluation metrics for Mistral-7B-Instruct were not available, it is known for its strong instruction-following capabilities, which contribute to the overall performance of the merged model.

Limitations

Despite the strengths of the merged model, it may inherit some limitations from its parent models. Potential issues include:

Biases: The model may reflect biases present in the training data of both parent models, which could affect the fairness and neutrality of its outputs.
Contextual Understanding: While the model excels in many areas, there may still be challenges in understanding highly nuanced or ambiguous prompts.

In summary, the Hermes-2-Pro-Mistral-7B-Mistral-7B-Instruct-v0.3-linear-merge represents a powerful tool for a wide range of NLP tasks, combining the best features of its parent models while also carrying forward some of their limitations.

Downloads last month: 54

GGUF

Model size

7B params

Architecture

llama

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support