MarinDNA m5.1 1B base model

MarinDNA m5.1 is a 1.12B-parameter, nucleotide-level causal language model developed with Marin. This is the final m5.1 base-model checkpoint at step 59,158 from run dna-bolinas-mix-v0.9-p1B-i24-exp135-zoonomia-m5.1-bef41e, released with the A 1B standard Transformer rivals Evo 2 40B on variant effect prediction blog post.

Model details

Field Value
Architecture Qwen3ForCausalLM-compatible decoder-only Transformer
Parameters 1,120,772,224
Layers 19
Hidden / intermediate size 1,920 / 7,680
Attention heads / KV heads 15 / 15
Context 256 tokens: one BOS token followed by up to 255 DNA bases
Checkpoint Final m5.1 checkpoint, step 59,158
Approximate token exposure 166.0B nucleotide tokens over the inherited training lineage
Stored weight dtype float32
License Apache-2.0

The source checkpoint is gs://marin-us-east5/checkpoints/dna-bolinas-mix-v0.9-p1B-i24-exp135-zoonomia-m5.1-bef41e/hf/step-59158. The commit-pinned training script defines the experiment.

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "marin-dna/marin-dna-exp135-m5.1"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

The float32 checkpoint is approximately 4.18 GiB. Convert or load it in a lower-precision dtype only if that is appropriate for your downstream use.

Tokenizer and input format

The bundled tokenizer is a case-insensitive, single-nucleotide tokenizer with seven tokens:

Token ID
[PAD] 0
[UNK] 1
[BOS] 2
a 3
c 4
g 5
t 6

Pass raw DNA strings containing A, C, G, and T without spaces or separators. The tokenizer lowercases input and automatically prepends [BOS]; the model has no EOS token and the tokenizer does not append one. Characters outside the four canonical bases map to [UNK]. Because BOS occupies one of the 256 model positions, inputs are limited to 255 DNA bases.

Training lineage and data

m5.1 is a continued-training lineage with approximately 166.0B total inherited and newly seen nucleotide tokens:

  1. approximately 42.0B inherited tokens from the pre-cooldown point of a uniform three-region mixture (CDS, upstream, and downstream);
  2. approximately 62.0B tokens from a continued uniform three-region mixture; and
  3. approximately 62.0B tokens from a uniform five-region mixture adding enhancer and ncRNA sequence.

The five-region inventory below describes the resources used across the m5.1 lineage. It does not mean that all five datasets contributed to every training phase.

Training datasets

Matched training-validation probes

These matched datasets were validation probes, not training data.

Downloads last month
1,678
Safetensors
Model size
1B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including marin-dna/marin-dna-exp135-m5.1