Hugging Face's logo

rubra-ai
/
Qwen2-7B-Instruct-GGUF

GGUF
English
Chinese
function-calling
tool-calling
agentic
rubra
conversational
Eval Results (legacy)
Model card Files
xet
 Community
How to use from
llama.cpp
Install (macOS, Linux) 
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf rubra-ai/Qwen2-7B-Instruct-GGUF:
# Run inference directly in the terminal:
llama cli -hf rubra-ai/Qwen2-7B-Instruct-GGUF:
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf rubra-ai/Qwen2-7B-Instruct-GGUF:
# Run inference directly in the terminal:
llama cli -hf rubra-ai/Qwen2-7B-Instruct-GGUF:
Use pre-built binary 
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf rubra-ai/Qwen2-7B-Instruct-GGUF:
# Run inference directly in the terminal:
./llama-cli -hf rubra-ai/Qwen2-7B-Instruct-GGUF:
Build from source code 
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf rubra-ai/Qwen2-7B-Instruct-GGUF:
# Run inference directly in the terminal:
./build/bin/llama-cli -hf rubra-ai/Qwen2-7B-Instruct-GGUF:
Use Docker 
docker model run hf.co/rubra-ai/Qwen2-7B-Instruct-GGUF:
Quick Links

Qwen2 7B Instruct GGUF

Original model: rubra-ai/Qwen2-7B-Instruct

Model description

The model is the result of further post-training Qwen/Qwen2-7B-Instruct. It is capable of complex multi-turn tool/function calling.

Training

The model was post-trained (freeze tuned & DPO) on a proprietary dataset consisting of diverse function calling, chat, and instruct data.

How to use

Refer to https://docs.rubra.ai/inference/llamacpp for usage. Feel free to ask/open issues up in our Github repo: https://github.com/rubra-ai/rubra

Limitations and Bias

While the model performs well on a wide range of tasks, it may still produce biased or incorrect outputs. Users should exercise caution and critical judgment when using the model in sensitive or high-stakes applications. The model's outputs are influenced by the data it was trained on, which may contain inherent biases.

Ethical Considerations

Users should ensure that the deployment of this model adheres to ethical guidelines and consider the potential societal impact of the generated text. Misuse of the model for generating harmful or misleading content is strongly discouraged.

Acknowledgements

We would like to thank Alibaba Cloud for the model.

Contact Information

For questions or comments about the model, please reach out to the rubra team.

Citation

If you use this work, please cite it as:

@misc {rubra_ai_2024,
    author       = { Sanjay Nadhavajhala and Yingbei Tong },
    title        = { Rubra-Qwen2-7B-Instruct },
    year         = 2024,
    url          = { https://huggingface.co/rubra-ai/Qwen2-7B-Instruct },
    doi          = { 10.57967/hf/2683 },
    publisher    = { Hugging Face }
}
Downloads last month
118
GGUF
Model size
9B params
Architecture
qwen2
Hardware compatibility
Log In to add your hardware

4-bit

5-bit

6-bit

8-bit

View +1 variant
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including rubra-ai/Qwen2-7B-Instruct-GGUF

Evaluation results