---
language: en
license: apache-2.0
base_model: gpt2
tags:
- text-generation
- open-mythos
- fine-web-edu
---

# OpenMythos Tiny (FineWeb-Edu)

This is a custom architecture 'OpenMythos' model (approx 17.1M parameters) trained on the FineWeb-Edu dataset.

## Usage

To use this model, you need to use `trust_remote_code=True` because it uses a custom recurrence-based architecture.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Danilosistu/open-mythos-tiny-fineweb")
model = AutoModelForCausalLM.from_pretrained(
    "Danilosistu/open-mythos-tiny-fineweb", 
    trust_remote_code=True
)

inputs = tokenizer("Once upon a time", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0]))
```