Update README.md

README.md

@@ -41,6 +41,49 @@ Write information about the nucleotide sequence.
 Information about location in the kaniwa chromosome: >lcl|Cp5
 ```
 
+## Usage
+
+### Basic inference
+
+```python
+from peft import AutoPeftModelForCausalLM
+from transformers import AutoTokenizer
+
+model = AutoPeftModelForCausalLM.from_pretrained("monsoon-nlp/llama3-biotokenpretrain-kaniwa", load_in_4bit=True).to("cuda")
+tokenizer = AutoTokenizer.from_pretrained("monsoon-nlp/llama3-biotokenpretrain-kaniwa")
+tokenizer.pad_token = tokenizer.eos_token # pad fix
+
+qed = "∎" # from math symbols, used in pretraining
+sequence = "".join([(qed + nt) for nt in "GCCTATAGTGTGTAGCTAATGAGCCTAGGTTATCGACCCTAATCT"])
+
+inputs = tokenizer(f"{prefix}{sequence}{annotation}", return_tensors="pt")
+outputs = model.generate(input_ids=inputs["input_ids"].to("cuda"), max_new_tokens=50)
+sample = tokenizer.batch_decode(outputs, skip_special_tokens=False)[0]
+```
+
+### LoRA finetuning on a new task
+
+```python
+from trl import SFTTrainer
+from unsloth import FastLanguageModel
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name = "monsoon-nlp/llama3-biotokenpretrain-kaniwa",
+    max_seq_length = 7_000, # max 6,000 bp for AgroNT tasks
+    dtype = None,
+    load_in_4bit = True,
+    resize_model_vocab=128260, # includes biotokens
+)
+tokenizer.pad_token = tokenizer.eos_token # pad fix
+
+trainer = SFTTrainer(
+    model = model,
+    tokenizer = tokenizer,
+...
+)
+```
+
+
 This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.