Danilosistu/open-mythos-tiny-fineweb

OpenMythos Tiny (FineWeb-Edu)

This is a custom architecture 'OpenMythos' model (approx 17.1M parameters) trained on the FineWeb-Edu dataset.

Usage

To use this model, you need to use trust_remote_code=True because it uses a custom recurrence-based architecture.

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Danilosistu/open-mythos-tiny-fineweb")
model = AutoModelForCausalLM.from_pretrained(
    "Danilosistu/open-mythos-tiny-fineweb", 
    trust_remote_code=True
)

inputs = tokenizer("Once upon a time", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0]))