---
tags:
- chemistry
- molecular-design
- transformer
- generative-model
- predictive-model
license: bsd-3-clause
datasets:
- GuacaMol
- ZINC
- MoleculeNet
gated: true
extra_gated_fields:
  Organization: text
  Intended use: text
  Contact person: text
  E-mail: text
  Country: country
  Date: date_picker
  I agree to use this model only for purposes that are non-malicious and ethically responsible: checkbox
  I have read and accept the BSD 3-Clause license: checkbox
---

# Hyformer

Hyformer is a joint transformer-based model that unifies a generative decoder with a predictive encoder. Depending on the task, Hyformer uses either a causal or a bidirectional mask, outputting token probabilities or predicted property values.

## Model Details
- **Paper:** [Synergistic Benefits of Joint Molecule Generation and Property Prediction](https://arxiv.org/abs/2504.16559)
- **Authors:** Adam Izdebski, Jan Olszewski, Pankhil Gawade, Krzysztof Koras, Serra Korkmaz, Valentin Rauscher, Jakub M. Tomczak, Ewa Szczurek
- **License:** BSD 3-Clause
- **Repository:** [https://github.com/szczurek-lab/hyformer](https://github.com/szczurek-lab/hyformer)

## Model checkpoints
- **[Hyformer_molecules_8M](https://huggingface.co/SzczurekLab/hyformer_molecules_8M):** Trained on GuacaMol dataset ([Brown et al., 2019](https://jcheminf.biomedcentral.com/articles/10.1186/s13321-019-0351-9))
- **[Hyformer_molecules_50M](https://huggingface.co/SzczurekLab/hyformer_molecules_50M):** Trained on 19M molecules from ZINC, ChEMBL, and other purchasable molecular datasets ([Zhou et al., 2023](https://openreview.net/forum?id=1pPpKc9wR0Y))
- **[Hyformer_peptides_34M](https://huggingface.co/SzczurekLab/hyformer_peptides_34M):** Trained on 3.5M general-purpose and antimicrobial peptides
- **[Hyformer_peptides_34M_MIC](https://huggingface.co/SzczurekLab/hyformer_peptides_34M_MIC):** `Hyformer_peptides_34M` jointly fine-tuned on minimal inhibitory concentration values (MIC) against E. coli bacteria

## Gated Access
This model is available with **gated access**. To request access, please use the Hugging Face gated request form.

## Citation
If you use this model, please cite:

```
@misc{izdebski2025synergisticbenefitsjointmolecule,
      title={Synergistic Benefits of Joint Molecule Generation and Property Prediction}, 
      author={Adam Izdebski and Jan Olszewski and Pankhil Gawade and Krzysztof Koras and Serra Korkmaz and Valentin Rauscher and Jakub M. Tomczak and Ewa Szczurek},
      year={2025},
      eprint={2504.16559},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2504.16559}, 
}
```

## References
- Brown, Nathan, et al. "GuacaMol: benchmarking models for de novo molecular design." Journal of chemical information and modeling, 2019.
- Zhou, Gengmo, et al. "Uni-mol: A universal 3d molecular representation learning framework." ICLR, 2023.