---
pipeline_tag: fill-mask
language: eng
license: apache-2.0
tags:
  - trimmed
library_name: transformers
base_model: answerdotai/ModernBERT-large
base_model_relation: quantized
datasets:
  - Who-find-it-first/fineweb-2-trimming
---

# ModernBERT-large-eng-32768

This model is a trimmed version of [answerdotai/ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large)
optimized for **English** via vocabulary reduction
([trimming method](https://huggingface.co/blog/introduction-to-trimming)).

## Model Statistics
| Metric | Original | Trimmed | Reduction |
|--------|----------|---------|-----------|
| **Vocabulary size** | 50,368 | 32,768 | **34.94%** |
| **Model size** | 395,881,664 params | 377,841,664 params | **4.56%** |

## Mining Dataset Statistics
- **Texts used for mining**: 200,000
- **Dataset**: [Who-find-it-first/fineweb-2-trimming](https://huggingface.co/datasets/Who-find-it-first/fineweb-2-trimming)

## Usage
```python
from transformers import AutoModelForMaskedLM, AutoTokenizer

model = AutoModelForMaskedLM.from_pretrained("lbourdois/ModernBERT-large-32768")
tokenizer = AutoTokenizer.from_pretrained("lbourdois/ModernBERT-large-32768")
```