ArcticSpeculator
Build the fastest OSS vllm-based speculative decoding system for your own model, using ArcticTraining and ArcticInference!
For more details about ArcticSpeculator and how to use it:
- ❄️ Using Arctic-Inference and Arctic-Training for improving real-world speculative decoding Performance (blog)
- 🚀 Getting started guide using ArcticTraining
See all of the speculators we have released via our Speculators Collection
- Downloads last month
- 34
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support