metadata
title: >-
The New Equation: Dense AdapterMoE — Dynamic Expert Routing on Consumer
Hardware — Hayula Research
authors:
- Hayula AI Lab
papers:
- paper.md
tags:
- adapter-moe
- expert-routing
- consumer-hardware
- dense
license: mit
The New Equation: Dense AdapterMoE — Dynamic Expert Routing on Consumer Hardware — Hayula Research
Hayula AI Lab
Abstract
Mixture-of-Experts models achieve efficient scaling by routing tokens to specialized sub-networks. However, traditional MoE requires training from scratch or expensive full-model fine-tuning. We propose Dense AdapterMoE: a hybrid architecture combining a frozen 7-8B base language model with 10-20 dynamically routed LoRA expert adapters, all loaded simultaneously on consumer hardware (Apple M2 Ultra, 192GB unified memory). Each expert adapter occupies only 37MB (2 million parameters at rank 8), a
Files
| File | Description |
|---|---|
paper.md |
Full paper (Markdown) |
README.md |
This model card |
Citation
@techreport{hayulalab2026newequationadaptermoe,
title={The New Equation: Dense AdapterMoE — Dynamic Expert Routing on Consumer Hardware — Hayula Research},
author={Hayula AI Lab},
year={2026},
url={https://huggingface.co/hayulalab/new-equation-adapter-moe-paper}
}
hayulalab — Open Source AI Research