Update README.md

README.md

@@ -1,36 +1,62 @@
----
-pipeline_tag: fill-mask
-language: eng
-license: apache-2.0
-tags:
-  - trimmed
-library_name: transformers
-base_model: answerdotai/ModernBERT-large
-base_model_relation: quantized
-datasets:
-  - Who-find-it-first/fineweb-2-trimming
----
-
-# ModernBERT-large-eng-16384
-
-This model is a trimmed version of [answerdotai/ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large)
-optimized for **English** via vocabulary reduction
-([trimming method](https://huggingface.co/blog/introduction-to-trimming)).
-
-## Model Statistics
-| Metric | Original | Trimmed | Reduction |
-|--------|----------|---------|-----------|
-| **Vocabulary size** | 50,368 | 16,384 | **67.47%** |
-| **Model size** | 395,881,664 params | 361,048,064 params | **8.80%** |
-
-## Mining Dataset Statistics
-- **Texts used for mining**: 200,000
-- **Dataset**: [Who-find-it-first/fineweb-2-trimming](https://huggingface.co/datasets/Who-find-it-first/fineweb-2-trimming)
-
-## Usage
-```python
-from transformers import AutoModelForMaskedLM, AutoTokenizer
-
-model = AutoModelForMaskedLM.from_pretrained("lbourdois/ModernBERT-large-16384")
-tokenizer = AutoTokenizer.from_pretrained("lbourdois/ModernBERT-large-16384")
-```
+---
+pipeline_tag: fill-mask
+language: en
+license: apache-2.0
+tags:
+  - trimmed
+library_name: transformers
+base_model: answerdotai/ModernBERT-large
+base_model_relation: quantized
+datasets:
+  - lbourdois/fineweb-2-trimming
+---
+
+# ModernBERT-large-16384
+This model is a **8.80% smaller** version of [answerdotai/ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large) optimized for **English** language via vocabulary size reduction using the [trimming](https://huggingface.co/blog/lbourdois/introduction-to-trimming) method.  
+This trimmed model should perform similarly to the original model with only 16,384 tokens and a much smaller memory footprint.
+
+## Model Statistics
+| Metric | Original | Trimmed | Reduction |
+|--------|----------|---------|-----------|
+| **Vocabulary size** | 50,368 | 16,384 | **67.47%** |
+| **Model size** | 395,881,664 params | 361,048,064 params | **8.80%** |
+
+![image](https://raw.githubusercontent.com/lbourdois/blog/refs/heads/master/assets/images/Trimming/ModernBERT-large-16384.png)
+
+## Mining Dataset Statistics
+- **Number of texts used for mining**: 200,000 texts  
+- **Dataset**: [lbourdois/fineweb-2-trimming](https://huggingface.co/datasets/lbourdois/fineweb-2-trimming)
+
+## Usage
+```python
+from transformers import AutoModel, AutoTokenizer
+
+model_name = "alphaedge-ai/ModernBERT-large-16384"
+model = AutoModel.from_pretrained(model_name)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+```
+
+## Citations
+
+#### ModernBERT
+```
+@misc{modernbert,
+      title={Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference}, 
+      author={Benjamin Warner and Antoine Chaffin and Benjamin Clavié and Orion Weller and Oskar Hallström and Said Taghadouini and Alexis Gallagher and Raja Biswas and Faisal Ladhak and Tom Aarsen and Nathan Cooper and Griffin Adams and Jeremy Howard and Iacopo Poli},
+      year={2024},
+      eprint={2412.13663},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2412.13663}, 
+}
+```
+
+#### Trimming blog post
+```
+@misc{hf_blogpost_trimming,
+      title={Introduction to Trimming}, 
+      author={Loïck BOURDOIS and Tom AARSEN and Bram VANROY and Christopher AKIKI and Woojun JUNG and Manuel ROMERO and Prithiv SAKTHI},
+      year={2026},
+      url={https://huggingface.co/blog/lbourdois/introduction-to-trimming}, 
+}
+```