Instructions to use RekaAI/reka-flash-3.1-rekaquant-q3_k_s with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RekaAI/reka-flash-3.1-rekaquant-q3_k_s with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="RekaAI/reka-flash-3.1-rekaquant-q3_k_s",
	filename="reka-flash-3.1-rekaquant-q3_k_s.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use RekaAI/reka-flash-3.1-rekaquant-q3_k_s with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S
# Run inference directly in the terminal:
llama cli -hf RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S
# Run inference directly in the terminal:
llama cli -hf RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S
# Run inference directly in the terminal:
./llama-cli -hf RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S
# Run inference directly in the terminal:
./build/bin/llama-cli -hf RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S

Use Docker

docker model run hf.co/RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S

LM Studio
Jan
Ollama
How to use RekaAI/reka-flash-3.1-rekaquant-q3_k_s with Ollama:
```
ollama run hf.co/RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S
```

Unsloth Studio

How to use RekaAI/reka-flash-3.1-rekaquant-q3_k_s with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for RekaAI/reka-flash-3.1-rekaquant-q3_k_s to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for RekaAI/reka-flash-3.1-rekaquant-q3_k_s to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for RekaAI/reka-flash-3.1-rekaquant-q3_k_s to start chatting

Atomic Chat new
Docker Model Runner
How to use RekaAI/reka-flash-3.1-rekaquant-q3_k_s with Docker Model Runner:
```
docker model run hf.co/RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S
```

Lemonade

How to use RekaAI/reka-flash-3.1-rekaquant-q3_k_s with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull RekaAI/reka-flash-3.1-rekaquant-q3_k_s:Q3_K_S

Run and chat with the model

lemonade run user.reka-flash-3.1-rekaquant-q3_k_s-Q3_K_S

List all available models

lemonade list

Reka Flash 3.1 (3.5 bit)

This repository corresponds to the quantized version of Reka Flash 3.1. It has been quantized using our Reka Quant method, which leverages calibrated error reduction and online self-distillation to reduce quantization loss. The GGUF corresponds to Q3_K_S quantization.

You can find the half-precision version here, and the Reka Quant quantization library here

Learn more about our quantization technology.

Quick Start

Reka Flash 3.1 Quantized is released in a llama.cpp-compatible Q3_K_S format. You may use any library compatible with GGUF to run the model.

Via llama.cpp

./llama-cli -hf rekaai/reka-flash-3.1-rekaquant-q3_k_s -p "Who are you?"

Model Details

Prompt Format

Reka Flash 3.1 uses cl100k_base tokenizer and adds no additional special tokens. Its prompt format is as follows:

human: this is round 1 prompt <sep> assistant: this is round 1 response <sep> ...

Generation should stop on seeing the string <sep> or seeing the special token <|endoftext|>. System prompt can be added by prepending to the first user round.

human: You are a friendly assistant blah ... this is round 1 user prompt <sep> assistant: this is round 1 response <sep> ...

For multi-round conversations, it is recommended to drop the reasoning traces in the previous assistant round to save tokens for the model to think. If you are using HF or vLLM, the built-in chat_template shall handle prompt formatting automatically.

Language Support

This model is primarily built for the English language, and you should consider this an English only model. However, the model is able to converse and understand other languages to some degree.

Downloads last month: 14

GGUF

Model size

21B params

Architecture

llama

Hardware compatibility

3-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support