---
library_name: transformers
license: apache-2.0
tags:
  - biology
  - genomics
  - plasmid
  - dna
  - causal-lm
  - synthetic-biology
language:
  - en
pipeline_tag: text-generation
---

# PlasmidLM-kmer6

A 19.3M parameter autoregressive language model for **plasmid DNA sequence generation**, trained on ~100K plasmid sequences from Addgene.

## Model Details

| Property | Value |
|---|---|
| Parameters | 19.3M |
| Architecture | Transformer decoder (dense MLP) |
| Hidden size | 384 |
| Layers | 10 |
| Attention heads | 8 |
| Intermediate size | 1,536 |
| Max sequence length | 16,384 tokens |
| Tokenizer | k-mer (k=6, stride=3) |
| Vocab size | 4,208 |

### Training

- **Data**: ~100K plasmid sequences from Addgene, tokenized with k-mer (k=6, stride=3)
- **Steps**: 65,000
- **Eval loss**: 0.129
- **Token accuracy**: 97.4%

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("McClain/PlasmidLM-kmer6", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("McClain/PlasmidLM-kmer6", trust_remote_code=True)

# Condition on antibiotic resistance + origin of replication
prompt = "<BOS><AMR_KANAMYCIN><ORI_COLE1><SEP>"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.8, do_sample=True, top_p=0.95)
print(tokenizer.decode(outputs[0].tolist()))
```

The model generates plasmid DNA sequences conditioned on functional annotations (antibiotic resistance markers, origins of replication) provided as special tokens in the prompt.

## Special Tokens

| Token | Purpose |
|---|---|
| `<BOS>` | Beginning of sequence |
| `<EOS>` | End of sequence |
| `<SEP>` | Separator between prompt annotations and DNA sequence |
| `<PAD>` | Padding |
| `<AMR_*>` | Antibiotic resistance markers (e.g., `<AMR_KANAMYCIN>`, `<AMR_AMPICILLIN>`) |
| `<ORI_*>` | Origins of replication (e.g., `<ORI_COLE1>`, `<ORI_P15A>`) |

## Citation

If you use this model, please cite:

```
@misc{thiel2026plasmidlm,
  title={PlasmidLM: Language Models for Plasmid DNA Generation},
  author={Thiel, McClain},
  year={2026}
}
```