---
language:
  - en
license: eupl-1.2
library_name: mlx
tags:
  - mlx
  - safetensors
  - gemma4
  - lethean
  - lem
  - text-generation
  - conversational
  - 4-bit
base_model:
  - google/gemma-4-26B-A4B-it
base_model_relation: quantized
pipeline_tag: text-generation
---

# Lemmy

A Gemma 4 26B MoE fine-tune by [Lethean Network](https://lthn.ai/lemmy).

EUPL-1.2 · Apache 2.0 base · [lthn.ai/lemmy](https://lthn.ai/lemmy)

## Use

### MLX

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("lthn/lemmy", revision="4bit")
response = generate(model, tokenizer, prompt="Hello", max_tokens=200)
```

### Ollama

```bash
# Coming soon
```

### HF Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("lthn/lemmy", revision="bf16-hf")
tokenizer = AutoTokenizer.from_pretrained("lthn/lemmy", revision="bf16-hf")
```

## Branches

### MLX

| Branch | Size |
|--------|------|
| `bf16` | 47G |
| `8bit` | 25G |
| `6bit` | 19G |
| `5bit` | 16G |
| `4bit` | 13G |
| `mxfp8` | 24G |
| `mxfp4` | 13G |
| `nvfp4` | 13G |

### GGUF

| Branch | Size |
|--------|------|
| `bf16-gguf` | Coming soon |
| `8bit-gguf` | Coming soon |
| `6bit-gguf` | Coming soon |
| `5bit-gguf` | Coming soon |
| `4bit-gguf` | Coming soon |

### HF Transformers

| Branch | Size |
|--------|------|
| `bf16-hf` | Coming soon |

## Base

[google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it)

## More

- [lthn.ai/lemmy](https://lthn.ai/lemmy)
- [Lethean Network](https://lthn.ai)
- [GitHub](https://github.com/dappcore)

## Licence

Training data and adapter: [EUPL-1.2](https://joinup.ec.europa.eu/collection/eupl/eupl-text-eupl-12)
Base model: [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)