output = llm(
"Once upon a time,",
max_tokens=512,
echo=True
)
print(output)Polaris GGUF Quants
This repository contains GGUF (GPT-Generated Unified Format) model files for the Polaris model.
These files were quantized using an importance matrix generated from the Polaris dataset for optimal performance. The imatrix file can be viewed on the Github for this repository (coming soon). The importance matrix was generated using 10k semi-randomly chosen examples from the Polaris Dataset
Associated Github
The Github for this project contains the required modelfile settings necessary to run this model with Ollama along with some of the scripts I used to create these imatrix quantizations.
Accreditation
All credit for the original Polaris model belongs to their team and their affiliated organizations. Thank you for all of your hard work, and open publishing of the models and research 😊, the POLARIS recipe will be absolutely crucial for edge LLM computing and if you haven't read their original paper I'd reccomend giving it a good read to truly appreciate it.
@misc{Polaris2025,
title = {POLARIS: A Post-Training Recipe for Scaling Reinforcement Learning on Advanced Reasoning Models},
url = {https://hkunlp.github.io/blog/2025/Polaris},
author = {An, Chenxin and Xie, Zhihui and Li, Xiaonan and Li, Lei and Zhang, Jun and Gong, Shansan and Zhong, Ming and Xu, Jingjing and Qiu, Xipeng and Wang, Mingxuan and Kong, Lingpeng}
year = {2025}
}
Ollama Integration
To use these models with Ollama, llmstudio, llama.cpp, etc. select your desired quantization level from the dropdown menu (e.g., Q4_K_M) and use the provided command.
Note: Replace Q4_K_M with the tag for the specific model version you wish to download.
ollama pull hf.co/dzur658/Polaris-4B-Preview-IQ-GGUF:Q4_K_M
- Downloads last month
- 275
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="dzur658/Polaris-4B-Preview-IQ-GGUF", filename="", )