Hugging Face's logo

LetheanNetwork
/
lemmy-mlx-8bit

Image-Text-to-Text
MLX
Safetensors
gemma4
apple-silicon
8bit
Mixture of Experts
mixture-of-experts
on-device
conversational
8-bit precision
Model card Files
xet
 Community
How to use from
Hermes Agent
Start the MLX server 
# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "LetheanNetwork/lemmy-mlx-8bit"
Configure Hermes 
# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default LetheanNetwork/lemmy-mlx-8bit
Run Hermes 
hermes
Quick Links

LetheanNetwork/lemmy-mlx-8bit

Gemma 4 26B A4B MoE in MLX format, 8-bit quantized, converted from LetheanNetwork/lemmy's bf16 safetensors via mlx_lm.convert. Higher-precision sibling of LetheanNetwork/lemmy-mlx (4-bit). For the LEK-merged variant see lthn/lemmy.

License

Apache 2.0, subject to the Gemma Terms of Use.

Downloads last month
3
Safetensors
Model size
25B params
Tensor type
BF16
·
U32
·
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
Image-Text-to-Text
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for LetheanNetwork/lemmy-mlx-8bit

Base model

google/gemma-4-26B-A4B
Finetuned
google/gemma-4-26B-A4B-it
Quantized
LetheanNetwork/lemmy
Quantized
(2)
this model