OLMoE (November 2024)
Collection
Artifacts for open mixture-of-experts language models. • 13 items • Updated • 31
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf allenai/OLMoE-1B-7B-0924-GGUF:# Run inference directly in the terminal:
llama-cli -hf allenai/OLMoE-1B-7B-0924-GGUF:Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf allenai/OLMoE-1B-7B-0924-GGUF:# Run inference directly in the terminal:
./llama-cli -hf allenai/OLMoE-1B-7B-0924-GGUF:Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf allenai/OLMoE-1B-7B-0924-GGUF:# Run inference directly in the terminal:
./build/bin/llama-cli -hf allenai/OLMoE-1B-7B-0924-GGUF:Use Docker
docker model run hf.co/allenai/OLMoE-1B-7B-0924-GGUF:Quick Links
GGUF version of https://huggingface.co/allenai/OLMoE-1B-7B-0924
@misc{muennighoff2024olmoeopenmixtureofexpertslanguage,
title={OLMoE: Open Mixture-of-Experts Language Models},
author={Niklas Muennighoff and Luca Soldaini and Dirk Groeneveld and Kyle Lo and Jacob Morrison and Sewon Min and Weijia Shi and Pete Walsh and Oyvind Tafjord and Nathan Lambert and Yuling Gu and Shane Arora and Akshita Bhagia and Dustin Schwenk and David Wadden and Alexander Wettig and Binyuan Hui and Tim Dettmers and Douwe Kiela and Ali Farhadi and Noah A. Smith and Pang Wei Koh and Amanpreet Singh and Hannaneh Hajishirzi},
year={2024},
eprint={2409.02060},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2409.02060},
}
- Downloads last month
- 321
Hardware compatibility
Log In to add your hardware
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for allenai/OLMoE-1B-7B-0924-GGUF
Base model
allenai/OLMoE-1B-7B-0924
Install from brew
# Start a local OpenAI-compatible server with a web UI: llama-server -hf allenai/OLMoE-1B-7B-0924-GGUF:# Run inference directly in the terminal: llama-cli -hf allenai/OLMoE-1B-7B-0924-GGUF: