hayulalab/swarm-of-specialists-paper

Swarm of Specialists: Can Multi-Model Ensembles of Fine-Tuned 7-8B Specialists Surpass Monolithic 70B+ Foundation Models? — Hayula Research

Hayula AI Lab

Abstract

The dominant paradigm in large language model deployment favors monolithic architectures: single large models (70B–400B+ parameters) trained on diverse data and tasked with general-purpose reasoning. This paper presents an alternative approach developed at Hayula Labs: ensembles of fine-tuned 7-8B parameter specialist models, each optimized for a narrow domain, coordinated by a lightweight routing layer.

Files

File	Description
paper.md	Full paper (Markdown)
README.md	This model card

Citation

@techreport{hayulalab2026swarmofspecialists,
    title={Swarm of Specialists: Can Multi-Model Ensembles of Fine-Tuned 7-8B Specialists Surpass Monolithic 70B+ Foundation Models? — Hayula Research},
    author={Hayula AI Lab},
    year={2026},
    url={https://huggingface.co/hayulalab/swarm-of-specialists-paper}
}

---

hayulalab — Open Source AI Research