---
title: "The New Equation: Dense AdapterMoE — Dynamic Expert Routing on Consumer Hardware — Hayula Research"
authors:
- Hayula AI Lab
papers:
- paper.md
tags:
- adapter-moe
- expert-routing
- consumer-hardware
- dense
license: mit
---

# The New Equation: Dense AdapterMoE — Dynamic Expert Routing on Consumer Hardware — Hayula Research

**Hayula AI Lab**

## Abstract

Mixture-of-Experts models achieve efficient scaling by routing tokens to specialized sub-networks. However, traditional MoE requires training from scratch or expensive full-model fine-tuning. We propose Dense AdapterMoE: a hybrid architecture combining a frozen 7-8B base language model with 10-20 dynamically routed LoRA expert adapters, all loaded simultaneously on consumer hardware (Apple M2 Ultra, 192GB unified memory). Each expert adapter occupies only 37MB (2 million parameters at rank 8), a

## Files

| File | Description |
|------|-------------|
| `paper.md` | Full paper (Markdown) |
| `README.md` | This model card |

## Citation

```bibtex
@techreport{hayulalab2026newequationadaptermoe,
    title={The New Equation: Dense AdapterMoE — Dynamic Expert Routing on Consumer Hardware — Hayula Research},
    author={Hayula AI Lab},
    year={2026},
    url={https://huggingface.co/hayulalab/new-equation-adapter-moe-paper}
}
```

---

*hayulalab — Open Source AI Research*