Update README.md

README.md

@@ -1,3 +1,60 @@
----
-license: mit
----
+---
+license: mit
+datasets:
+- jonghyunlee/ZINC15
+---
+
+# 🧪 Mol-GPT2; long context, pretrained with ZINC-15
+
+This repository hosts a GPT-2-based model for generating SMILES strings, trained on the ZINC 15 dataset. The model follows the architecture and hyperparameter setup of MolGPT (Bagal et al., 2021), and has been fine-tuned to generate valid molecular representations with high accuracy.
+This model has longer context length (256), whereas the previous model has maximum context lenght 128. 
+
+---
+
+## 🔧 Model Architecture
+
+GPT2Config(
+    vocab_size=10_000,
+    n_positions=256,
+    n_ctx=256,
+    n_embd=256,
+    n_layer=8,
+    n_head=8,
+    resid_pdrop=0.1,
+    embd_pdrop=0.1,
+    attn_pdrop=0.1,
+)
+
+- Pretrained with fp16 precision on 2× H100 GPUs  
+- Batch size: 1,024  
+- Max steps: 100,000  
+- Warmup steps: 10,000  
+- Evaluation every 10,000 steps
+
+---
+
+## 📊 Performance
+
+| Dataset/Metric       | This Model | Short Context | MolGPT Baseline |
+|----------------------|------------|---------------|-----------------|
+| ZINC15 Validity      | 99.76%     | 99.68%        | N/A             |
+| MOSES Validity       | N/A        | N/A           | 99.4%           |
+| GuacaMol Validity    | N/A        | N/A           | 98.1%           |
+
+---
+
+## 🚀 Usage Example
+
+```
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+# Load tokenizer and model
+tokenizer = AutoTokenizer.from_pretrained("your-username/smiles-tokenizer")
+model = AutoModelForCausalLM.from_pretrained("your-username/molgpt-lite")
+
+# Generate molecules
+input_ids = tokenizer("CC(=O)OC1=CC=CC=C1C(=O)O", return_tensors="pt").input_ids
+outputs = model.generate(input_ids, max_length=256, do_sample=True, top_k=50)
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```