Instructions to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B")
model = PeftModel.from_pretrained(base_model, "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1")

Transformers

How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1

SGLang

How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1 with Docker Model Runner:
```
docker model run hf.co/1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1
```

1jamesthompson1 commited on 9 days ago

Commit

89868f7

verified ·

1 Parent(s): 8dff6b0

Upload folder using huggingface_hub

Browse files

Files changed (13) hide show

.gitattributes +1 -0
README.md +149 -0
adapter_config.json +50 -0
adapter_model.safetensors +3 -0
chat_template.jinja +154 -0
environment.json +82 -0
finetune_config.json +25 -0
processor_config.json +60 -0
tokenizer.json +3 -0
tokenizer_config.json +32 -0
training.log +16 -0
training_args.bin +3 -0
training_log.json +484 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,149 @@

+---
+base_model: Qwen/Qwen3.5-9B
+library_name: peft
+model_name: cluster_1
+tags:
+  - base_model:adapter:Qwen/Qwen3.5-9B
+  - lora
+  - sft
+  - transformers
+  - trl
+  - sampled_response
+  - cluster_1
+license: cc-by-sa-4.0
+pipeline_tag: text-generation
+datasets:
+  - 1jamesthompson1/wvs-nz-value-alignment
+---
+# Qwen3.5-9B LoRA — Sampled Response, Cluster 1
+This model is a LoRA fine-tune of [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)  as part of the
+[AIML589 project](https://github.com/1jamesthompson1/AIML589).
+This adapter is licensed under CC BY-SA 4.0.
+## Dataset
+Fine-tuned on the **sampled_response** config of the
+[wvs-nz-value-alignment](https://huggingface.co/datasets/1jamesthompson1/wvs-nz-value-alignment)
+dataset, **cluster_1** subpopulation.
+Part of the [wvs-nz-value-alignment](https://huggingface.co/collections/wvs-nz-value-alignment) collection.
+**GPU:** NVIDIA RTX 6000 Ada Generation · **Training time:** 18m 8s
+## Training hyperparameters
+| Parameter | Value |
+|-----------|-------|
+| LoRA rank | 16 |
+| LoRA alpha | 32 |
+| LoRA dropout | 0.05 |
+| DoRA | False |
+| Learning rate | 0.0002 |
+| Batch size | 4 |
+| Gradient accumulation | 4 |
+| Epochs | 3 |
+| Max seq length | 1024 |
+| Warmup ratio | 0.1 |
+| Dtype | bf16 |
+## Training log
+```
+{"loss": 4.2828514099121096, "grad_norm": 3.3195607662200928, "learning_rate": 7.82608695652174e-05, "entropy": 1.1838731363415718, "mean_token_accuracy": 0.452268467284739, "num_tokens": 24645.0, "epoch": 0.13245033112582782, "step": 10}
+{"eval_loss": 0.5712648034095764, "eval_runtime": 8.7607, "eval_samples_per_second": 17.122, "eval_steps_per_second": 4.338, "eval_entropy": 0.5471524494258981, "eval_mean_token_accuracy": 0.8050595286645388, "eval_num_tokens": 24645.0, "epoch": 0.13245033112582782, "step": 10}
+{"loss": 0.46926422119140626, "grad_norm": 3.0499935150146484, "learning_rate": 0.00016521739130434784, "entropy": 0.4832126975059509, "mean_token_accuracy": 0.8076777458190918, "num_tokens": 48985.0, "epoch": 0.26490066225165565, "step": 20}
+{"eval_loss": 0.5358471870422363, "eval_runtime": 8.8684, "eval_samples_per_second": 16.914, "eval_steps_per_second": 4.285, "eval_entropy": 0.4200292041427211, "eval_mean_token_accuracy": 0.7469924860878995, "eval_num_tokens": 48985.0, "epoch": 0.26490066225165565, "step": 20}
+{"loss": 0.39472386837005613, "grad_norm": 2.067092180252075, "learning_rate": 0.00019414634146341464, "entropy": 0.3605545647442341, "mean_token_accuracy": 0.8380664870142936, "num_tokens": 73226.0, "epoch": 0.3973509933774834, "step": 30}
+{"eval_loss": 0.5321611166000366, "eval_runtime": 8.8949, "eval_samples_per_second": 16.864, "eval_steps_per_second": 4.272, "eval_entropy": 0.36939919249791847, "eval_mean_token_accuracy": 0.8029448617445795, "eval_num_tokens": 73226.0, "epoch": 0.3973509933774834, "step": 30}
+{"loss": 0.39537787437438965, "grad_norm": 1.270966649055481, "learning_rate": 0.00018439024390243903, "entropy": 0.3851592864841223, "mean_token_accuracy": 0.8292971581220627, "num_tokens": 97170.0, "epoch": 0.5298013245033113, "step": 40}
+{"eval_loss": 0.512241542339325, "eval_runtime": 8.856, "eval_samples_per_second": 16.938, "eval_steps_per_second": 4.291, "eval_entropy": 0.3763781021300115, "eval_mean_token_accuracy": 0.7811560207291653, "eval_num_tokens": 97170.0, "epoch": 0.5298013245033113, "step": 40}
+{"loss": 0.3449979066848755, "grad_norm": 1.178625226020813, "learning_rate": 0.00017463414634146342, "entropy": 0.3603811550885439, "mean_token_accuracy": 0.8657324358820915, "num_tokens": 121386.0, "epoch": 0.6622516556291391, "step": 50}
+{"eval_loss": 0.6070794463157654, "eval_runtime": 8.9125, "eval_samples_per_second": 16.83, "eval_steps_per_second": 4.264, "eval_entropy": 0.36585210116678163, "eval_mean_token_accuracy": 0.7500313332206324, "eval_num_tokens": 121386.0, "epoch": 0.6622516556291391, "step": 50}
+{"loss": 0.30911884307861326, "grad_norm": 2.8266186714172363, "learning_rate": 0.00016487804878048782, "entropy": 0.2615261286497116, "mean_token_accuracy": 0.8826999247074128, "num_tokens": 145580.0, "epoch": 0.7947019867549668, "step": 60}
+{"eval_loss": 0.7182475924491882, "eval_runtime": 8.8882, "eval_samples_per_second": 16.876, "eval_steps_per_second": 4.275, "eval_entropy": 0.2628652969081151, "eval_mean_token_accuracy": 0.7899749420191112, "eval_num_tokens": 145580.0, "epoch": 0.7947019867549668, "step": 60}
+{"loss": 0.24180982112884522, "grad_norm": 1.6042029857635498, "learning_rate": 0.0001551219512195122, "entropy": 0.2673916194587946, "mean_token_accuracy": 0.911859568953514, "num_tokens": 170643.0, "epoch": 0.9271523178807947, "step": 70}
+{"eval_loss": 0.821165919303894, "eval_runtime": 8.8971, "eval_samples_per_second": 16.859, "eval_steps_per_second": 4.271, "eval_entropy": 0.27867767418195544, "eval_mean_token_accuracy": 0.7433740669175198, "eval_num_tokens": 170643.0, "epoch": 0.9271523178807947, "step": 70}
+{"loss": 0.22081174850463867, "grad_norm": 2.7884373664855957, "learning_rate": 0.0001453658536585366, "entropy": 0.2026200779762707, "mean_token_accuracy": 0.9230157701592696, "num_tokens": 193746.0, "epoch": 1.0529801324503312, "step": 80}
+{"eval_loss": 0.8150429725646973, "eval_runtime": 8.9086, "eval_samples_per_second": 16.838, "eval_steps_per_second": 4.266, "eval_entropy": 0.22414806843000024, "eval_mean_token_accuracy": 0.8047775735980586, "eval_num_tokens": 193746.0, "epoch": 1.0529801324503312, "step": 80}
+{"loss": 0.14040154218673706, "grad_norm": 1.3541489839553833, "learning_rate": 0.000135609756097561, "entropy": 0.16504043247550726, "mean_token_accuracy": 0.9462000757455826, "num_tokens": 218576.0, "epoch": 1.185430463576159, "step": 90}
+{"eval_loss": 0.8339773416519165, "eval_runtime": 8.973, "eval_samples_per_second": 16.717, "eval_steps_per_second": 4.235, "eval_entropy": 0.2065272086060998, "eval_mean_token_accuracy": 0.7746240653489765, "eval_num_tokens": 218576.0, "epoch": 1.185430463576159, "step": 90}
+{"loss": 0.12467809915542602, "grad_norm": 4.511893272399902, "learning_rate": 0.00012585365853658536, "entropy": 0.10107915503322147, "mean_token_accuracy": 0.9587799295783043, "num_tokens": 243344.0, "epoch": 1.3178807947019868, "step": 100}
+{"eval_loss": 0.7112035155296326, "eval_runtime": 8.9282, "eval_samples_per_second": 16.801, "eval_steps_per_second": 4.256, "eval_entropy": 0.15474749381016745, "eval_mean_token_accuracy": 0.8157424864016081, "eval_num_tokens": 243344.0, "epoch": 1.3178807947019868, "step": 100}
+{"loss": 0.12197786569595337, "grad_norm": 1.6361427307128906, "learning_rate": 0.00011609756097560975, "entropy": 0.07562960358336568, "mean_token_accuracy": 0.9517370820045471, "num_tokens": 267786.0, "epoch": 1.4503311258278146, "step": 110}
+{"eval_loss": 1.0470056533813477, "eval_runtime": 8.9724, "eval_samples_per_second": 16.718, "eval_steps_per_second": 4.235, "eval_entropy": 0.14818870221681305, "eval_mean_token_accuracy": 0.7478383512873399, "eval_num_tokens": 267786.0, "epoch": 1.4503311258278146, "step": 110}
+{"loss": 0.10809319019317627, "grad_norm": 1.6775206327438354, "learning_rate": 0.00010634146341463416, "entropy": 0.08924289270071313, "mean_token_accuracy": 0.9604727029800415, "num_tokens": 292317.0, "epoch": 1.5827814569536423, "step": 120}
+{"eval_loss": 1.0492334365844727, "eval_runtime": 8.8734, "eval_samples_per_second": 16.904, "eval_steps_per_second": 4.282, "eval_entropy": 0.14410611607232376, "eval_mean_token_accuracy": 0.8007205567861858, "eval_num_tokens": 292317.0, "epoch": 1.5827814569536423, "step": 120}
+{"loss": 0.03908039927482605, "grad_norm": 2.2686848640441895, "learning_rate": 9.658536585365854e-05, "entropy": 0.06353283888893202, "mean_token_accuracy": 0.9815119966864586, "num_tokens": 316521.0, "epoch": 1.7152317880794703, "step": 130}
+{"eval_loss": 1.4305915832519531, "eval_runtime": 8.8602, "eval_samples_per_second": 16.93, "eval_steps_per_second": 4.289, "eval_entropy": 0.0935905459689246, "eval_mean_token_accuracy": 0.7892387258379083, "eval_num_tokens": 316521.0, "epoch": 1.7152317880794703, "step": 130}
+{"loss": 0.04100652039051056, "grad_norm": 5.218770980834961, "learning_rate": 8.682926829268293e-05, "entropy": 0.03381892999459524, "mean_token_accuracy": 0.984375512599945, "num_tokens": 340919.0, "epoch": 1.847682119205298, "step": 140}
+{"eval_loss": 1.5555816888809204, "eval_runtime": 8.7739, "eval_samples_per_second": 17.096, "eval_steps_per_second": 4.331, "eval_entropy": 0.07794101885673099, "eval_mean_token_accuracy": 0.8050282001495361, "eval_num_tokens": 340919.0, "epoch": 1.847682119205298, "step": 140}
+{"loss": 0.03830587565898895, "grad_norm": 0.13190802931785583, "learning_rate": 7.707317073170732e-05, "entropy": 0.026670162624213845, "mean_token_accuracy": 0.9837620735168457, "num_tokens": 364991.0, "epoch": 1.980132450331126, "step": 150}
+{"eval_loss": 1.2507771253585815, "eval_runtime": 8.8746, "eval_samples_per_second": 16.902, "eval_steps_per_second": 4.282, "eval_entropy": 0.0817602070930757, "eval_mean_token_accuracy": 0.7987468728893682, "eval_num_tokens": 364991.0, "epoch": 1.980132450331126, "step": 150}
+{"loss": 0.07177287340164185, "grad_norm": 2.2161507606506348, "learning_rate": 6.731707317073171e-05, "entropy": 0.01958650479240245, "mean_token_accuracy": 0.9884510479475322, "num_tokens": 387642.0, "epoch": 2.1059602649006623, "step": 160}
+{"eval_loss": 1.3437883853912354, "eval_runtime": 8.8478, "eval_samples_per_second": 16.953, "eval_steps_per_second": 4.295, "eval_entropy": 0.06764962662648606, "eval_mean_token_accuracy": 0.7787750677058571, "eval_num_tokens": 387642.0, "epoch": 2.1059602649006623, "step": 160}
+{"loss": 0.009113987535238266, "grad_norm": 0.039817001670598984, "learning_rate": 5.756097560975609e-05, "entropy": 0.01661088205728447, "mean_token_accuracy": 0.9937208637595176, "num_tokens": 412538.0, "epoch": 2.23841059602649, "step": 170}
+{"eval_loss": 1.5150843858718872, "eval_runtime": 8.7479, "eval_samples_per_second": 17.147, "eval_steps_per_second": 4.344, "eval_entropy": 0.06736165528357226, "eval_mean_token_accuracy": 0.7647556442963449, "eval_num_tokens": 412538.0, "epoch": 2.23841059602649, "step": 170}
+{"loss": 0.003525285795331001, "grad_norm": 0.00853405799716711, "learning_rate": 4.7804878048780485e-05, "entropy": 0.00830267872515833, "mean_token_accuracy": 0.9980769231915474, "num_tokens": 437055.0, "epoch": 2.370860927152318, "step": 180}
+{"eval_loss": 1.644209384918213, "eval_runtime": 8.7784, "eval_samples_per_second": 17.087, "eval_steps_per_second": 4.329, "eval_entropy": 0.06734515197259545, "eval_mean_token_accuracy": 0.7691416097314734, "eval_num_tokens": 437055.0, "epoch": 2.370860927152318, "step": 180}
+{"loss": 0.001157384179532528, "grad_norm": 0.09711030125617981, "learning_rate": 3.804878048780488e-05, "entropy": 0.004432732032000786, "mean_token_accuracy": 1.0, "num_tokens": 461493.0, "epoch": 2.5033112582781456, "step": 190}
+{"eval_loss": 1.7609623670578003, "eval_runtime": 8.7515, "eval_samples_per_second": 17.14, "eval_steps_per_second": 4.342, "eval_entropy": 0.06280427154553381, "eval_mean_token_accuracy": 0.7669486277981808, "eval_num_tokens": 461493.0, "epoch": 2.5033112582781456, "step": 190}
+{"loss": 0.0030027683824300766, "grad_norm": 0.012400347739458084, "learning_rate": 2.8292682926829267e-05, "entropy": 0.004392967079911614, "mean_token_accuracy": 0.997916667163372, "num_tokens": 485528.0, "epoch": 2.6357615894039736, "step": 200}
+{"eval_loss": 1.8287278413772583, "eval_runtime": 8.7245, "eval_samples_per_second": 17.193, "eval_steps_per_second": 4.356, "eval_entropy": 0.06022026199015703, "eval_mean_token_accuracy": 0.7735275751666019, "eval_num_tokens": 485528.0, "epoch": 2.6357615894039736, "step": 200}
+{"loss": 0.0011992882937192917, "grad_norm": 0.004404825158417225, "learning_rate": 1.853658536585366e-05, "entropy": 0.0030429765494773163, "mean_token_accuracy": 1.0, "num_tokens": 510081.0, "epoch": 2.7682119205298013, "step": 210}
+{"eval_loss": 1.8526947498321533, "eval_runtime": 8.6297, "eval_samples_per_second": 17.382, "eval_steps_per_second": 4.403, "eval_entropy": 0.057951420073789596, "eval_mean_token_accuracy": 0.7669486277981808, "eval_num_tokens": 510081.0, "epoch": 2.7682119205298013, "step": 210}
+{"loss": 0.007075458765029907, "grad_norm": 0.029240623116493225, "learning_rate": 8.780487804878048e-06, "entropy": 0.006888014055220992, "mean_token_accuracy": 0.9928314402699471, "num_tokens": 534422.0, "epoch": 2.9006622516556293, "step": 220}
+{"eval_loss": 1.8728781938552856, "eval_runtime": 8.6887, "eval_samples_per_second": 17.264, "eval_steps_per_second": 4.374, "eval_entropy": 0.05773907397892081, "eval_mean_token_accuracy": 0.7735275751666019, "eval_num_tokens": 534422.0, "epoch": 2.9006622516556293, "step": 220}
+{"eval_loss": 1.882822871208191, "eval_runtime": 8.6056, "eval_samples_per_second": 17.43, "eval_steps_per_second": 4.416, "eval_entropy": 0.05690596842377042, "eval_mean_token_accuracy": 0.7669486277981808, "eval_num_tokens": 552555.0, "epoch": 3.0, "step": 228}
+{"train_runtime": 1087.8226, "train_samples_per_second": 3.326, "train_steps_per_second": 0.21, "total_flos": 3.178031783683891e+16, "train_loss": 0.3232832907233387, "epoch": 3.0, "step": 228}
+```
+## Environment
+| Package | Version |
+|---------|---------|
+| torch | 2.13.0 |
+| transformers | 5.14.1 |
+| trl | 1.9.2 |
+| datasets | 5.0.1 |
+| accelerate | 1.14.0 |
+| python-dotenv | 1.2.2 |
+| peft | 0.20.0 |
+| bitsandbytes | 0.50.0 |
+| huggingface-hub | ? |
+| jinja2 | ? |
+| torchvision | 0.28.0 |
+| pillow | 12.3.0 |
+## Intended use
+This adapter is intended for **research purposes only** as part of the
+[AIML589 project](https://github.com/1jamesthompson1/AIML589), which
+investigates value alignment of LLMs with New Zealand population
+distributions from the World Values Survey.
+### Out-of-scope
+This model has not been safety-tuned for general-purpose deployment.
+It should not be used in production systems, for making decisions about
+people, or in contexts where reliability and safety are critical.
+## Limitations and biases
+- Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
+- The training data reflects the values of those who responded to the
+  survey and may not represent all New Zealanders.
+- LoRA adapters are subject to the limitations and biases of the base
+  model ([Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)).

adapter_config.json ADDED Viewed

	@@ -0,0 +1,50 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "k_proj",
+    "o_proj",
+    "v_proj",
+    "gate_proj",
+    "q_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:60200bf98cf5290229c415d7e0379bdcdbe4220e3c750be1c4a052309a6a6762
+size 116425880

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,154 @@

+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

environment.json ADDED Viewed

	@@ -0,0 +1,82 @@

+{
+  "Jinja2": "3.1.6",
+  "MarkupSafe": "3.0.3",
+  "PyYAML": "6.0.3",
+  "Pygments": "2.20.0",
+  "accelerate": "1.14.0",
+  "aiohappyeyeballs": "2.7.1",
+  "aiohttp": "3.14.3",
+  "aiosignal": "1.4.0",
+  "annotated-doc": "0.0.5",
+  "anyio": "4.14.2",
+  "attrs": "26.1.0",
+  "bitsandbytes": "0.50.0",
+  "certifi": "2026.7.22",
+  "charset-normalizer": "3.4.9",
+  "click": "8.4.2",
+  "cuda-bindings": "13.3.1",
+  "cuda-pathfinder": "1.6.0",
+  "cuda-toolkit": "13.0.3.0",
+  "datasets": "5.0.1",
+  "dill": "0.4.1",
+  "filelock": "3.32.0",
+  "frozenlist": "1.8.0",
+  "fsspec": "2026.6.0",
+  "h11": "0.16.0",
+  "hf-xet": "1.5.2",
+  "httpcore": "1.0.9",
+  "httpx": "0.28.1",
+  "huggingface_hub": "1.25.1",
+  "idna": "3.18",
+  "markdown-it-py": "4.2.0",
+  "mdurl": "0.1.2",
+  "mpmath": "1.3.0",
+  "multidict": "6.7.1",
+  "multiprocess": "0.70.19",
+  "networkx": "3.6.1",
+  "numpy": "2.5.1",
+  "nvidia-cublas": "13.1.1.3",
+  "nvidia-cuda-cupti": "13.0.85",
+  "nvidia-cuda-nvrtc": "13.0.88",
+  "nvidia-cuda-runtime": "13.0.96",
+  "nvidia-cudnn-cu13": "9.20.0.48",
+  "nvidia-cufft": "12.0.0.61",
+  "nvidia-cufile": "1.15.1.6",
+  "nvidia-curand": "10.4.0.35",
+  "nvidia-cusolver": "12.0.4.66",
+  "nvidia-cusparse": "12.6.3.3",
+  "nvidia-cusparselt-cu13": "0.8.1",
+  "nvidia-nccl-cu13": "2.29.7",
+  "nvidia-nvjitlink": "13.3.33",
+  "nvidia-nvshmem-cu13": "3.4.5",
+  "nvidia-nvtx": "13.0.85",
+  "packaging": "26.2",
+  "pandas": "3.0.5",
+  "peft": "0.20.0",
+  "pillow": "12.3.0",
+  "propcache": "0.5.2",
+  "psutil": "7.2.2",
+  "pyarrow": "25.0.0",
+  "python-dateutil": "2.9.0.post0",
+  "python-dotenv": "1.2.2",
+  "regex": "2026.7.19",
+  "requests": "2.34.2",
+  "rich": "15.0.0",
+  "safetensors": "0.8.0",
+  "setuptools": "83.0.0",
+  "shellingham": "1.5.4",
+  "six": "1.17.0",
+  "sympy": "1.14.0",
+  "tokenizers": "0.22.2",
+  "torch": "2.13.0",
+  "torchvision": "0.28.0",
+  "tqdm": "4.70.0",
+  "transformers": "5.14.1",
+  "triton": "3.7.1",
+  "trl": "1.9.2",
+  "typer": "0.27.0",
+  "typing_extensions": "4.16.0",
+  "urllib3": "2.7.0",
+  "xxhash": "3.8.1",
+  "yarl": "1.24.5"
+}

finetune_config.json ADDED Viewed

	@@ -0,0 +1,25 @@

+{
+  "model": "Qwen/Qwen3.5-9B",
+  "model_sha": "c202236235762e1c871ad0ccb60c8ee5ba337b9a",
+  "dataset": "sampled_response",
+  "dataset_sha": "38f42265e06d60afd0a5d2d45808202e60e95103",
+  "subpopulation": "cluster_1",
+  "gpu": "NVIDIA RTX 6000 Ada Generation",
+  "library_name": "peft",
+  "pipeline_tag": "text-generation",
+  "timestamp": "2026-08-05T16:17:06.134064",
+  "hyperparameters": {
+    "lora_r": 16,
+    "lora_alpha": 32,
+    "lora_dropout": 0.05,
+    "dora": false,
+    "quantization": null,
+    "lr": 0.0002,
+    "num_epochs": 3,
+    "batch_size": 4,
+    "gradient_accumulation_steps": 4,
+    "max_seq_length": 1024,
+    "warmup_ratio": 0.1,
+    "dtype": "bf16"
+  }
+}

processor_config.json ADDED Viewed

	@@ -0,0 +1,60 @@

+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "merge_size": 2,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 16777216,
+      "shortest_edge": 65536
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 2,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 25165824,
+      "shortest_edge": 4096
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,32 @@

+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

training.log ADDED Viewed

	@@ -0,0 +1,16 @@

+logging to output/sampled_response/cluster_1/training.log
+[sft] loading tokenizer...
+[sft] loading model...
+[sft] tokenizing 1206 examples...
+[sft] tokenizing validation split for eval...
+[data] loaded 150 validation examples (config=sampled_response, subpop=cluster_1)
+[sft] eval split: 150 examples
+[sft] starting training...
+[sft] saving...
+    saved to output/sampled_response/cluster_1
+    gpu: NVIDIA RTX 6000 Ada Generation  time: 1088s
+    training log -> output/sampled_response/cluster_1/training_log.json
+    run metadata -> output/sampled_response/cluster_1/finetune_config.json
+    environment -> output/sampled_response/cluster_1/environment.json
+    README -> output/sampled_response/cluster_1/README.md
+[upload] uploading output/sampled_response/cluster_1 to https://huggingface.co/1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_1...

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:267e9d55cd978f8b4f4cf7d686302a32267c4b2ed4007e83fb81e831a5529b3a
+size 5713

training_log.json ADDED Viewed

	@@ -0,0 +1,484 @@

+[
+  {
+    "loss": 4.2828514099121096,
+    "grad_norm": 3.3195607662200928,
+    "learning_rate": 7.82608695652174e-05,
+    "entropy": 1.1838731363415718,
+    "mean_token_accuracy": 0.452268467284739,
+    "num_tokens": 24645.0,
+    "epoch": 0.13245033112582782,
+    "step": 10
+  },
+  {
+    "eval_loss": 0.5712648034095764,
+    "eval_runtime": 8.7607,
+    "eval_samples_per_second": 17.122,
+    "eval_steps_per_second": 4.338,
+    "eval_entropy": 0.5471524494258981,
+    "eval_mean_token_accuracy": 0.8050595286645388,
+    "eval_num_tokens": 24645.0,
+    "epoch": 0.13245033112582782,
+    "step": 10
+  },
+  {
+    "loss": 0.46926422119140626,
+    "grad_norm": 3.0499935150146484,
+    "learning_rate": 0.00016521739130434784,
+    "entropy": 0.4832126975059509,
+    "mean_token_accuracy": 0.8076777458190918,
+    "num_tokens": 48985.0,
+    "epoch": 0.26490066225165565,
+    "step": 20
+  },
+  {
+    "eval_loss": 0.5358471870422363,
+    "eval_runtime": 8.8684,
+    "eval_samples_per_second": 16.914,
+    "eval_steps_per_second": 4.285,
+    "eval_entropy": 0.4200292041427211,
+    "eval_mean_token_accuracy": 0.7469924860878995,
+    "eval_num_tokens": 48985.0,
+    "epoch": 0.26490066225165565,
+    "step": 20
+  },
+  {
+    "loss": 0.39472386837005613,
+    "grad_norm": 2.067092180252075,
+    "learning_rate": 0.00019414634146341464,
+    "entropy": 0.3605545647442341,
+    "mean_token_accuracy": 0.8380664870142936,
+    "num_tokens": 73226.0,
+    "epoch": 0.3973509933774834,
+    "step": 30
+  },
+  {
+    "eval_loss": 0.5321611166000366,
+    "eval_runtime": 8.8949,
+    "eval_samples_per_second": 16.864,
+    "eval_steps_per_second": 4.272,
+    "eval_entropy": 0.36939919249791847,
+    "eval_mean_token_accuracy": 0.8029448617445795,
+    "eval_num_tokens": 73226.0,
+    "epoch": 0.3973509933774834,
+    "step": 30
+  },
+  {
+    "loss": 0.39537787437438965,
+    "grad_norm": 1.270966649055481,
+    "learning_rate": 0.00018439024390243903,
+    "entropy": 0.3851592864841223,
+    "mean_token_accuracy": 0.8292971581220627,
+    "num_tokens": 97170.0,
+    "epoch": 0.5298013245033113,
+    "step": 40
+  },
+  {
+    "eval_loss": 0.512241542339325,
+    "eval_runtime": 8.856,
+    "eval_samples_per_second": 16.938,
+    "eval_steps_per_second": 4.291,
+    "eval_entropy": 0.3763781021300115,
+    "eval_mean_token_accuracy": 0.7811560207291653,
+    "eval_num_tokens": 97170.0,
+    "epoch": 0.5298013245033113,
+    "step": 40
+  },
+  {
+    "loss": 0.3449979066848755,
+    "grad_norm": 1.178625226020813,
+    "learning_rate": 0.00017463414634146342,
+    "entropy": 0.3603811550885439,
+    "mean_token_accuracy": 0.8657324358820915,
+    "num_tokens": 121386.0,
+    "epoch": 0.6622516556291391,
+    "step": 50
+  },
+  {
+    "eval_loss": 0.6070794463157654,
+    "eval_runtime": 8.9125,
+    "eval_samples_per_second": 16.83,
+    "eval_steps_per_second": 4.264,
+    "eval_entropy": 0.36585210116678163,
+    "eval_mean_token_accuracy": 0.7500313332206324,
+    "eval_num_tokens": 121386.0,
+    "epoch": 0.6622516556291391,
+    "step": 50
+  },
+  {
+    "loss": 0.30911884307861326,
+    "grad_norm": 2.8266186714172363,
+    "learning_rate": 0.00016487804878048782,
+    "entropy": 0.2615261286497116,
+    "mean_token_accuracy": 0.8826999247074128,
+    "num_tokens": 145580.0,
+    "epoch": 0.7947019867549668,
+    "step": 60
+  },
+  {
+    "eval_loss": 0.7182475924491882,
+    "eval_runtime": 8.8882,
+    "eval_samples_per_second": 16.876,
+    "eval_steps_per_second": 4.275,
+    "eval_entropy": 0.2628652969081151,
+    "eval_mean_token_accuracy": 0.7899749420191112,
+    "eval_num_tokens": 145580.0,
+    "epoch": 0.7947019867549668,
+    "step": 60
+  },
+  {
+    "loss": 0.24180982112884522,
+    "grad_norm": 1.6042029857635498,
+    "learning_rate": 0.0001551219512195122,
+    "entropy": 0.2673916194587946,
+    "mean_token_accuracy": 0.911859568953514,
+    "num_tokens": 170643.0,
+    "epoch": 0.9271523178807947,
+    "step": 70
+  },
+  {
+    "eval_loss": 0.821165919303894,
+    "eval_runtime": 8.8971,
+    "eval_samples_per_second": 16.859,
+    "eval_steps_per_second": 4.271,
+    "eval_entropy": 0.27867767418195544,
+    "eval_mean_token_accuracy": 0.7433740669175198,
+    "eval_num_tokens": 170643.0,
+    "epoch": 0.9271523178807947,
+    "step": 70
+  },
+  {
+    "loss": 0.22081174850463867,
+    "grad_norm": 2.7884373664855957,
+    "learning_rate": 0.0001453658536585366,
+    "entropy": 0.2026200779762707,
+    "mean_token_accuracy": 0.9230157701592696,
+    "num_tokens": 193746.0,
+    "epoch": 1.0529801324503312,
+    "step": 80
+  },
+  {
+    "eval_loss": 0.8150429725646973,
+    "eval_runtime": 8.9086,
+    "eval_samples_per_second": 16.838,
+    "eval_steps_per_second": 4.266,
+    "eval_entropy": 0.22414806843000024,
+    "eval_mean_token_accuracy": 0.8047775735980586,
+    "eval_num_tokens": 193746.0,
+    "epoch": 1.0529801324503312,
+    "step": 80
+  },
+  {
+    "loss": 0.14040154218673706,
+    "grad_norm": 1.3541489839553833,
+    "learning_rate": 0.000135609756097561,
+    "entropy": 0.16504043247550726,
+    "mean_token_accuracy": 0.9462000757455826,
+    "num_tokens": 218576.0,
+    "epoch": 1.185430463576159,
+    "step": 90
+  },
+  {
+    "eval_loss": 0.8339773416519165,
+    "eval_runtime": 8.973,
+    "eval_samples_per_second": 16.717,
+    "eval_steps_per_second": 4.235,
+    "eval_entropy": 0.2065272086060998,
+    "eval_mean_token_accuracy": 0.7746240653489765,
+    "eval_num_tokens": 218576.0,
+    "epoch": 1.185430463576159,
+    "step": 90
+  },
+  {
+    "loss": 0.12467809915542602,
+    "grad_norm": 4.511893272399902,
+    "learning_rate": 0.00012585365853658536,
+    "entropy": 0.10107915503322147,
+    "mean_token_accuracy": 0.9587799295783043,
+    "num_tokens": 243344.0,
+    "epoch": 1.3178807947019868,
+    "step": 100
+  },
+  {
+    "eval_loss": 0.7112035155296326,
+    "eval_runtime": 8.9282,
+    "eval_samples_per_second": 16.801,
+    "eval_steps_per_second": 4.256,
+    "eval_entropy": 0.15474749381016745,
+    "eval_mean_token_accuracy": 0.8157424864016081,
+    "eval_num_tokens": 243344.0,
+    "epoch": 1.3178807947019868,
+    "step": 100
+  },
+  {
+    "loss": 0.12197786569595337,
+    "grad_norm": 1.6361427307128906,
+    "learning_rate": 0.00011609756097560975,
+    "entropy": 0.07562960358336568,
+    "mean_token_accuracy": 0.9517370820045471,
+    "num_tokens": 267786.0,
+    "epoch": 1.4503311258278146,
+    "step": 110
+  },
+  {
+    "eval_loss": 1.0470056533813477,
+    "eval_runtime": 8.9724,
+    "eval_samples_per_second": 16.718,
+    "eval_steps_per_second": 4.235,
+    "eval_entropy": 0.14818870221681305,
+    "eval_mean_token_accuracy": 0.7478383512873399,
+    "eval_num_tokens": 267786.0,
+    "epoch": 1.4503311258278146,
+    "step": 110
+  },
+  {
+    "loss": 0.10809319019317627,
+    "grad_norm": 1.6775206327438354,
+    "learning_rate": 0.00010634146341463416,
+    "entropy": 0.08924289270071313,
+    "mean_token_accuracy": 0.9604727029800415,
+    "num_tokens": 292317.0,
+    "epoch": 1.5827814569536423,
+    "step": 120
+  },
+  {
+    "eval_loss": 1.0492334365844727,
+    "eval_runtime": 8.8734,
+    "eval_samples_per_second": 16.904,
+    "eval_steps_per_second": 4.282,
+    "eval_entropy": 0.14410611607232376,
+    "eval_mean_token_accuracy": 0.8007205567861858,
+    "eval_num_tokens": 292317.0,
+    "epoch": 1.5827814569536423,
+    "step": 120
+  },
+  {
+    "loss": 0.03908039927482605,
+    "grad_norm": 2.2686848640441895,
+    "learning_rate": 9.658536585365854e-05,
+    "entropy": 0.06353283888893202,
+    "mean_token_accuracy": 0.9815119966864586,
+    "num_tokens": 316521.0,
+    "epoch": 1.7152317880794703,
+    "step": 130
+  },
+  {
+    "eval_loss": 1.4305915832519531,
+    "eval_runtime": 8.8602,
+    "eval_samples_per_second": 16.93,
+    "eval_steps_per_second": 4.289,
+    "eval_entropy": 0.0935905459689246,
+    "eval_mean_token_accuracy": 0.7892387258379083,
+    "eval_num_tokens": 316521.0,
+    "epoch": 1.7152317880794703,
+    "step": 130
+  },
+  {
+    "loss": 0.04100652039051056,
+    "grad_norm": 5.218770980834961,
+    "learning_rate": 8.682926829268293e-05,
+    "entropy": 0.03381892999459524,
+    "mean_token_accuracy": 0.984375512599945,
+    "num_tokens": 340919.0,
+    "epoch": 1.847682119205298,
+    "step": 140
+  },
+  {
+    "eval_loss": 1.5555816888809204,
+    "eval_runtime": 8.7739,
+    "eval_samples_per_second": 17.096,
+    "eval_steps_per_second": 4.331,
+    "eval_entropy": 0.07794101885673099,
+    "eval_mean_token_accuracy": 0.8050282001495361,
+    "eval_num_tokens": 340919.0,
+    "epoch": 1.847682119205298,
+    "step": 140
+  },
+  {
+    "loss": 0.03830587565898895,
+    "grad_norm": 0.13190802931785583,
+    "learning_rate": 7.707317073170732e-05,
+    "entropy": 0.026670162624213845,
+    "mean_token_accuracy": 0.9837620735168457,
+    "num_tokens": 364991.0,
+    "epoch": 1.980132450331126,
+    "step": 150
+  },
+  {
+    "eval_loss": 1.2507771253585815,
+    "eval_runtime": 8.8746,
+    "eval_samples_per_second": 16.902,
+    "eval_steps_per_second": 4.282,
+    "eval_entropy": 0.0817602070930757,
+    "eval_mean_token_accuracy": 0.7987468728893682,
+    "eval_num_tokens": 364991.0,
+    "epoch": 1.980132450331126,
+    "step": 150
+  },
+  {
+    "loss": 0.07177287340164185,
+    "grad_norm": 2.2161507606506348,
+    "learning_rate": 6.731707317073171e-05,
+    "entropy": 0.01958650479240245,
+    "mean_token_accuracy": 0.9884510479475322,
+    "num_tokens": 387642.0,
+    "epoch": 2.1059602649006623,
+    "step": 160
+  },
+  {
+    "eval_loss": 1.3437883853912354,
+    "eval_runtime": 8.8478,
+    "eval_samples_per_second": 16.953,
+    "eval_steps_per_second": 4.295,
+    "eval_entropy": 0.06764962662648606,
+    "eval_mean_token_accuracy": 0.7787750677058571,
+    "eval_num_tokens": 387642.0,
+    "epoch": 2.1059602649006623,
+    "step": 160
+  },
+  {
+    "loss": 0.009113987535238266,
+    "grad_norm": 0.039817001670598984,
+    "learning_rate": 5.756097560975609e-05,
+    "entropy": 0.01661088205728447,
+    "mean_token_accuracy": 0.9937208637595176,
+    "num_tokens": 412538.0,
+    "epoch": 2.23841059602649,
+    "step": 170
+  },
+  {
+    "eval_loss": 1.5150843858718872,
+    "eval_runtime": 8.7479,
+    "eval_samples_per_second": 17.147,
+    "eval_steps_per_second": 4.344,
+    "eval_entropy": 0.06736165528357226,
+    "eval_mean_token_accuracy": 0.7647556442963449,
+    "eval_num_tokens": 412538.0,
+    "epoch": 2.23841059602649,
+    "step": 170
+  },
+  {
+    "loss": 0.003525285795331001,
+    "grad_norm": 0.00853405799716711,
+    "learning_rate": 4.7804878048780485e-05,
+    "entropy": 0.00830267872515833,
+    "mean_token_accuracy": 0.9980769231915474,
+    "num_tokens": 437055.0,
+    "epoch": 2.370860927152318,
+    "step": 180
+  },
+  {
+    "eval_loss": 1.644209384918213,
+    "eval_runtime": 8.7784,
+    "eval_samples_per_second": 17.087,
+    "eval_steps_per_second": 4.329,
+    "eval_entropy": 0.06734515197259545,
+    "eval_mean_token_accuracy": 0.7691416097314734,
+    "eval_num_tokens": 437055.0,
+    "epoch": 2.370860927152318,
+    "step": 180
+  },
+  {
+    "loss": 0.001157384179532528,
+    "grad_norm": 0.09711030125617981,
+    "learning_rate": 3.804878048780488e-05,
+    "entropy": 0.004432732032000786,
+    "mean_token_accuracy": 1.0,
+    "num_tokens": 461493.0,
+    "epoch": 2.5033112582781456,
+    "step": 190
+  },
+  {
+    "eval_loss": 1.7609623670578003,
+    "eval_runtime": 8.7515,
+    "eval_samples_per_second": 17.14,
+    "eval_steps_per_second": 4.342,
+    "eval_entropy": 0.06280427154553381,
+    "eval_mean_token_accuracy": 0.7669486277981808,
+    "eval_num_tokens": 461493.0,
+    "epoch": 2.5033112582781456,
+    "step": 190
+  },
+  {
+    "loss": 0.0030027683824300766,
+    "grad_norm": 0.012400347739458084,
+    "learning_rate": 2.8292682926829267e-05,
+    "entropy": 0.004392967079911614,
+    "mean_token_accuracy": 0.997916667163372,
+    "num_tokens": 485528.0,
+    "epoch": 2.6357615894039736,
+    "step": 200
+  },
+  {
+    "eval_loss": 1.8287278413772583,
+    "eval_runtime": 8.7245,
+    "eval_samples_per_second": 17.193,
+    "eval_steps_per_second": 4.356,
+    "eval_entropy": 0.06022026199015703,
+    "eval_mean_token_accuracy": 0.7735275751666019,
+    "eval_num_tokens": 485528.0,
+    "epoch": 2.6357615894039736,
+    "step": 200
+  },
+  {
+    "loss": 0.0011992882937192917,
+    "grad_norm": 0.004404825158417225,
+    "learning_rate": 1.853658536585366e-05,
+    "entropy": 0.0030429765494773163,
+    "mean_token_accuracy": 1.0,
+    "num_tokens": 510081.0,
+    "epoch": 2.7682119205298013,
+    "step": 210
+  },
+  {
+    "eval_loss": 1.8526947498321533,
+    "eval_runtime": 8.6297,
+    "eval_samples_per_second": 17.382,
+    "eval_steps_per_second": 4.403,
+    "eval_entropy": 0.057951420073789596,
+    "eval_mean_token_accuracy": 0.7669486277981808,
+    "eval_num_tokens": 510081.0,
+    "epoch": 2.7682119205298013,
+    "step": 210
+  },
+  {
+    "loss": 0.007075458765029907,
+    "grad_norm": 0.029240623116493225,
+    "learning_rate": 8.780487804878048e-06,
+    "entropy": 0.006888014055220992,
+    "mean_token_accuracy": 0.9928314402699471,
+    "num_tokens": 534422.0,
+    "epoch": 2.9006622516556293,
+    "step": 220
+  },
+  {
+    "eval_loss": 1.8728781938552856,
+    "eval_runtime": 8.6887,
+    "eval_samples_per_second": 17.264,
+    "eval_steps_per_second": 4.374,
+    "eval_entropy": 0.05773907397892081,
+    "eval_mean_token_accuracy": 0.7735275751666019,
+    "eval_num_tokens": 534422.0,
+    "epoch": 2.9006622516556293,
+    "step": 220
+  },
+  {
+    "eval_loss": 1.882822871208191,
+    "eval_runtime": 8.6056,
+    "eval_samples_per_second": 17.43,
+    "eval_steps_per_second": 4.416,
+    "eval_entropy": 0.05690596842377042,
+    "eval_mean_token_accuracy": 0.7669486277981808,
+    "eval_num_tokens": 552555.0,
+    "epoch": 3.0,
+    "step": 228
+  },
+  {
+    "train_runtime": 1087.8226,
+    "train_samples_per_second": 3.326,
+    "train_steps_per_second": 0.21,
+    "total_flos": 3.178031783683891e+16,
+    "train_loss": 0.3232832907233387,
+    "epoch": 3.0,
+    "step": 228
+  }
+]