Instructions to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B")
model = PeftModel.from_pretrained(base_model, "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0")

Transformers

How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0

SGLang

How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0 with Docker Model Runner:
```
docker model run hf.co/1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0
```

1jamesthompson1 commited on 10 days ago

Commit

c6f7c46

verified ·

1 Parent(s): d2ffa6a

Upload folder using huggingface_hub

Browse files

Files changed (13) hide show

.gitattributes +1 -0
README.md +149 -0
adapter_config.json +50 -0
adapter_model.safetensors +3 -0
chat_template.jinja +154 -0
environment.json +82 -0
finetune_config.json +25 -0
processor_config.json +60 -0
tokenizer.json +3 -0
tokenizer_config.json +32 -0
training.log +16 -0
training_args.bin +3 -0
training_log.json +484 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,149 @@

+---
+base_model: Qwen/Qwen3.5-9B
+library_name: peft
+model_name: cluster_0
+tags:
+  - base_model:adapter:Qwen/Qwen3.5-9B
+  - lora
+  - sft
+  - transformers
+  - trl
+  - sampled_response
+  - cluster_0
+license: cc-by-sa-4.0
+pipeline_tag: text-generation
+datasets:
+  - 1jamesthompson1/wvs-nz-value-alignment
+---
+# Qwen3.5-9B LoRA — Sampled Response, Cluster 0
+This model is a LoRA fine-tune of [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)  as part of the
+[AIML589 project](https://github.com/1jamesthompson1/AIML589).
+This adapter is licensed under CC BY-SA 4.0.
+## Dataset
+Fine-tuned on the **sampled_response** config of the
+[wvs-nz-value-alignment](https://huggingface.co/datasets/1jamesthompson1/wvs-nz-value-alignment)
+dataset, **cluster_0** subpopulation.
+Part of the [wvs-nz-value-alignment](https://huggingface.co/collections/wvs-nz-value-alignment) collection.
+**GPU:** NVIDIA RTX 6000 Ada Generation · **Training time:** 18m 22s
+## Training hyperparameters
+| Parameter | Value |
+|-----------|-------|
+| LoRA rank | 16 |
+| LoRA alpha | 32 |
+| LoRA dropout | 0.05 |
+| DoRA | False |
+| Learning rate | 0.0002 |
+| Batch size | 4 |
+| Gradient accumulation | 4 |
+| Epochs | 3 |
+| Max seq length | 1024 |
+| Warmup ratio | 0.1 |
+| Dtype | bf16 |
+## Training log
+```
+{"loss": 4.099612808227539, "grad_norm": 3.3916518688201904, "learning_rate": 7.82608695652174e-05, "entropy": 1.0828558593988418, "mean_token_accuracy": 0.44957960266619923, "num_tokens": 24658.0, "epoch": 0.13245033112582782, "step": 10}
+{"eval_loss": 0.5499903559684753, "eval_runtime": 9.3737, "eval_samples_per_second": 16.002, "eval_steps_per_second": 4.054, "eval_entropy": 0.5737218025483584, "eval_mean_token_accuracy": 0.7952850923726433, "eval_num_tokens": 24658.0, "epoch": 0.13245033112582782, "step": 10}
+{"loss": 0.4331226825714111, "grad_norm": 3.621782064437866, "learning_rate": 0.00016521739130434784, "entropy": 0.4598923083394766, "mean_token_accuracy": 0.8161924198269844, "num_tokens": 48994.0, "epoch": 0.26490066225165565, "step": 20}
+{"eval_loss": 0.5726626515388489, "eval_runtime": 9.3836, "eval_samples_per_second": 15.985, "eval_steps_per_second": 4.05, "eval_entropy": 0.3104277225701432, "eval_mean_token_accuracy": 0.8032894793309664, "eval_num_tokens": 48994.0, "epoch": 0.26490066225165565, "step": 20}
+{"loss": 0.37956321239471436, "grad_norm": 3.6681301593780518, "learning_rate": 0.00019414634146341464, "entropy": 0.35528335012495516, "mean_token_accuracy": 0.8529162302613258, "num_tokens": 73216.0, "epoch": 0.3973509933774834, "step": 30}
+{"eval_loss": 0.4902452528476715, "eval_runtime": 9.6515, "eval_samples_per_second": 15.542, "eval_steps_per_second": 3.937, "eval_entropy": 0.3591178828164151, "eval_mean_token_accuracy": 0.8244517586733165, "eval_num_tokens": 73216.0, "epoch": 0.3973509933774834, "step": 30}
+{"loss": 0.3535017728805542, "grad_norm": 1.529909372329712, "learning_rate": 0.00018439024390243903, "entropy": 0.31258961334824564, "mean_token_accuracy": 0.8600528433918952, "num_tokens": 97165.0, "epoch": 0.5298013245033113, "step": 40}
+{"eval_loss": 0.5773032903671265, "eval_runtime": 8.8699, "eval_samples_per_second": 16.911, "eval_steps_per_second": 4.284, "eval_entropy": 0.40491867006609317, "eval_mean_token_accuracy": 0.7638627878929439, "eval_num_tokens": 97165.0, "epoch": 0.5298013245033113, "step": 40}
+{"loss": 0.29483020305633545, "grad_norm": 1.4113677740097046, "learning_rate": 0.00017463414634146342, "entropy": 0.3084935720078647, "mean_token_accuracy": 0.8778989180922508, "num_tokens": 121376.0, "epoch": 0.6622516556291391, "step": 50}
+{"eval_loss": 0.5696188807487488, "eval_runtime": 8.921, "eval_samples_per_second": 16.814, "eval_steps_per_second": 4.26, "eval_entropy": 0.29807096403582317, "eval_mean_token_accuracy": 0.8039630376978925, "eval_num_tokens": 121376.0, "epoch": 0.6622516556291391, "step": 50}
+{"loss": 0.26492722034454347, "grad_norm": 2.0661537647247314, "learning_rate": 0.00016487804878048782, "entropy": 0.268452774733305, "mean_token_accuracy": 0.8974904179573059, "num_tokens": 145557.0, "epoch": 0.7947019867549668, "step": 60}
+{"eval_loss": 0.8559898138046265, "eval_runtime": 8.9149, "eval_samples_per_second": 16.826, "eval_steps_per_second": 4.263, "eval_entropy": 0.2022110644458352, "eval_mean_token_accuracy": 0.8000626611082178, "eval_num_tokens": 145557.0, "epoch": 0.7947019867549668, "step": 60}
+{"loss": 0.18335200548171998, "grad_norm": 3.771090269088745, "learning_rate": 0.0001551219512195122, "entropy": 0.12369259365368634, "mean_token_accuracy": 0.9281293749809265, "num_tokens": 170605.0, "epoch": 0.9271523178807947, "step": 70}
+{"eval_loss": 0.8269464373588562, "eval_runtime": 8.912, "eval_samples_per_second": 16.831, "eval_steps_per_second": 4.264, "eval_entropy": 0.23079313014290834, "eval_mean_token_accuracy": 0.7807487523869464, "eval_num_tokens": 170605.0, "epoch": 0.9271523178807947, "step": 70}
+{"loss": 0.18180986642837524, "grad_norm": 2.6767499446868896, "learning_rate": 0.0001453658536585366, "entropy": 0.16967114948324466, "mean_token_accuracy": 0.9279846925484506, "num_tokens": 193719.0, "epoch": 1.0529801324503312, "step": 80}
+{"eval_loss": 0.8880820870399475, "eval_runtime": 8.8086, "eval_samples_per_second": 17.029, "eval_steps_per_second": 4.314, "eval_entropy": 0.2405679833119441, "eval_mean_token_accuracy": 0.7817669220660862, "eval_num_tokens": 193719.0, "epoch": 1.0529801324503312, "step": 80}
+{"loss": 0.11491576433181763, "grad_norm": 0.69949871301651, "learning_rate": 0.000135609756097561, "entropy": 0.14066557474434377, "mean_token_accuracy": 0.9485534891486168, "num_tokens": 218542.0, "epoch": 1.185430463576159, "step": 90}
+{"eval_loss": 0.7975587844848633, "eval_runtime": 8.8612, "eval_samples_per_second": 16.928, "eval_steps_per_second": 4.288, "eval_entropy": 0.19119705641826026, "eval_mean_token_accuracy": 0.7988721861651069, "eval_num_tokens": 218542.0, "epoch": 1.185430463576159, "step": 90}
+{"loss": 0.08289253115653991, "grad_norm": 2.014707565307617, "learning_rate": 0.00012585365853658536, "entropy": 0.05604892138508148, "mean_token_accuracy": 0.9736187428236007, "num_tokens": 243302.0, "epoch": 1.3178807947019868, "step": 100}
+{"eval_loss": 1.1870805025100708, "eval_runtime": 8.8416, "eval_samples_per_second": 16.965, "eval_steps_per_second": 4.298, "eval_entropy": 0.114802976807512, "eval_mean_token_accuracy": 0.8206453684129213, "eval_num_tokens": 243302.0, "epoch": 1.3178807947019868, "step": 100}
+{"loss": 0.06358585357666016, "grad_norm": 1.5396313667297363, "learning_rate": 0.00011609756097560975, "entropy": 0.04575981950329151, "mean_token_accuracy": 0.9785756185650826, "num_tokens": 267729.0, "epoch": 1.4503311258278146, "step": 110}
+{"eval_loss": 0.8791955709457397, "eval_runtime": 8.8863, "eval_samples_per_second": 16.88, "eval_steps_per_second": 4.276, "eval_entropy": 0.17355985742515737, "eval_mean_token_accuracy": 0.8070958703756332, "eval_num_tokens": 267729.0, "epoch": 1.4503311258278146, "step": 110}
+{"loss": 0.049860316514968875, "grad_norm": 0.678949773311615, "learning_rate": 0.00010634146341463416, "entropy": 0.07773116882308387, "mean_token_accuracy": 0.9833469331264496, "num_tokens": 292279.0, "epoch": 1.5827814569536423, "step": 120}
+{"eval_loss": 1.1025115251541138, "eval_runtime": 8.9449, "eval_samples_per_second": 16.769, "eval_steps_per_second": 4.248, "eval_entropy": 0.1594012568683339, "eval_mean_token_accuracy": 0.7980889779956717, "eval_num_tokens": 292279.0, "epoch": 1.5827814569536423, "step": 120}
+{"loss": 0.07376678586006165, "grad_norm": 2.6976561546325684, "learning_rate": 9.658536585365854e-05, "entropy": 0.0441209072858328, "mean_token_accuracy": 0.9765854984521866, "num_tokens": 316471.0, "epoch": 1.7152317880794703, "step": 130}
+{"eval_loss": 1.2516794204711914, "eval_runtime": 8.9032, "eval_samples_per_second": 16.848, "eval_steps_per_second": 4.268, "eval_entropy": 0.12382908019782535, "eval_mean_token_accuracy": 0.770441735261365, "eval_num_tokens": 316471.0, "epoch": 1.7152317880794703, "step": 130}
+{"loss": 0.02282872200012207, "grad_norm": 0.8916032314300537, "learning_rate": 8.682926829268293e-05, "entropy": 0.0408805181941716, "mean_token_accuracy": 0.9929233521223069, "num_tokens": 340856.0, "epoch": 1.847682119205298, "step": 140}
+{"eval_loss": 1.273831844329834, "eval_runtime": 8.873, "eval_samples_per_second": 16.905, "eval_steps_per_second": 4.283, "eval_entropy": 0.11421615798501175, "eval_mean_token_accuracy": 0.7660557698262366, "eval_num_tokens": 340856.0, "epoch": 1.847682119205298, "step": 140}
+{"loss": 0.020523886382579803, "grad_norm": 2.7703590393066406, "learning_rate": 7.707317073170732e-05, "entropy": 0.028159727145248326, "mean_token_accuracy": 0.9912698417901993, "num_tokens": 364928.0, "epoch": 1.980132450331126, "step": 150}
+{"eval_loss": 1.5218627452850342, "eval_runtime": 8.8524, "eval_samples_per_second": 16.945, "eval_steps_per_second": 4.293, "eval_entropy": 0.09264942740800937, "eval_mean_token_accuracy": 0.7557487519163835, "eval_num_tokens": 364928.0, "epoch": 1.980132450331126, "step": 150}
+{"loss": 0.015184570848941804, "grad_norm": 0.04658397287130356, "learning_rate": 6.731707317073171e-05, "entropy": 0.016722216987737307, "mean_token_accuracy": 0.9981203000796469, "num_tokens": 387571.0, "epoch": 2.1059602649006623, "step": 160}
+{"eval_loss": 1.6605678796768188, "eval_runtime": 8.8568, "eval_samples_per_second": 16.936, "eval_steps_per_second": 4.29, "eval_entropy": 0.07781053681920687, "eval_mean_token_accuracy": 0.770441735261365, "eval_num_tokens": 387571.0, "epoch": 2.1059602649006623, "step": 160}
+{"loss": 0.006011705845594406, "grad_norm": 0.013418244197964668, "learning_rate": 5.756097560975609e-05, "entropy": 0.007514824125610176, "mean_token_accuracy": 0.9968627452850342, "num_tokens": 412476.0, "epoch": 2.23841059602649, "step": 170}
+{"eval_loss": 1.763867974281311, "eval_runtime": 8.8357, "eval_samples_per_second": 16.977, "eval_steps_per_second": 4.301, "eval_entropy": 0.07184799665475639, "eval_mean_token_accuracy": 0.7737312089455756, "eval_num_tokens": 412476.0, "epoch": 2.23841059602649, "step": 170}
+{"loss": 0.004858419671654702, "grad_norm": 0.14898061752319336, "learning_rate": 4.7804878048780485e-05, "entropy": 0.004944811350287637, "mean_token_accuracy": 0.9980769231915474, "num_tokens": 436969.0, "epoch": 2.370860927152318, "step": 180}
+{"eval_loss": 1.827174425125122, "eval_runtime": 8.8372, "eval_samples_per_second": 16.974, "eval_steps_per_second": 4.3, "eval_entropy": 0.06969891959229962, "eval_mean_token_accuracy": 0.7720864721034703, "eval_num_tokens": 436969.0, "epoch": 2.370860927152318, "step": 180}
+{"loss": 0.005421416833996773, "grad_norm": 0.08094702661037445, "learning_rate": 3.804878048780488e-05, "entropy": 0.004511333805567119, "mean_token_accuracy": 0.9966666668653488, "num_tokens": 461400.0, "epoch": 2.5033112582781456, "step": 190}
+{"eval_loss": 1.859080195426941, "eval_runtime": 8.9117, "eval_samples_per_second": 16.832, "eval_steps_per_second": 4.264, "eval_entropy": 0.06829966303944514, "eval_mean_token_accuracy": 0.7753759457876808, "eval_num_tokens": 461400.0, "epoch": 2.5033112582781456, "step": 190}
+{"loss": 0.005886765569448471, "grad_norm": 0.007371403276920319, "learning_rate": 2.8292682926829267e-05, "entropy": 0.006164468544739066, "mean_token_accuracy": 0.9951255336403847, "num_tokens": 485443.0, "epoch": 2.6357615894039736, "step": 200}
+{"eval_loss": 1.8880680799484253, "eval_runtime": 8.8924, "eval_samples_per_second": 16.868, "eval_steps_per_second": 4.273, "eval_entropy": 0.06679329828732175, "eval_mean_token_accuracy": 0.7753759457876808, "eval_num_tokens": 485443.0, "epoch": 2.6357615894039736, "step": 200}
+{"loss": 0.0009710748679935932, "grad_norm": 0.0006506519857794046, "learning_rate": 1.853658536585366e-05, "entropy": 0.002073692464909982, "mean_token_accuracy": 1.0, "num_tokens": 509990.0, "epoch": 2.7682119205298013, "step": 210}
+{"eval_loss": 1.9050503969192505, "eval_runtime": 8.8469, "eval_samples_per_second": 16.955, "eval_steps_per_second": 4.295, "eval_entropy": 0.06654788765100468, "eval_mean_token_accuracy": 0.7737312089455756, "eval_num_tokens": 509990.0, "epoch": 2.7682119205298013, "step": 210}
+{"loss": 0.01160859763622284, "grad_norm": 0.004122971091419458, "learning_rate": 8.780487804878048e-06, "entropy": 0.00959118189784931, "mean_token_accuracy": 0.9908364906907081, "num_tokens": 534335.0, "epoch": 2.9006622516556293, "step": 220}
+{"eval_loss": 1.9109818935394287, "eval_runtime": 8.8226, "eval_samples_per_second": 17.002, "eval_steps_per_second": 4.307, "eval_entropy": 0.06608304418458781, "eval_mean_token_accuracy": 0.7737312089455756, "eval_num_tokens": 534335.0, "epoch": 2.9006622516556293, "step": 220}
+{"eval_loss": 1.9071061611175537, "eval_runtime": 8.9501, "eval_samples_per_second": 16.76, "eval_steps_per_second": 4.246, "eval_entropy": 0.06653332771269274, "eval_mean_token_accuracy": 0.7737312089455756, "eval_num_tokens": 552465.0, "epoch": 3.0, "step": 228}
+{"train_runtime": 1102.1217, "train_samples_per_second": 3.283, "train_steps_per_second": 0.207, "total_flos": 3.178328110793933e+16, "train_loss": 0.2925478722695915, "epoch": 3.0, "step": 228}
+```
+## Environment
+| Package | Version |
+|---------|---------|
+| torch | 2.13.0 |
+| transformers | 5.14.1 |
+| trl | 1.9.2 |
+| datasets | 5.0.1 |
+| accelerate | 1.14.0 |
+| python-dotenv | 1.2.2 |
+| peft | 0.20.0 |
+| bitsandbytes | 0.50.0 |
+| huggingface-hub | ? |
+| jinja2 | ? |
+| torchvision | 0.28.0 |
+| pillow | 12.3.0 |
+## Intended use
+This adapter is intended for **research purposes only** as part of the
+[AIML589 project](https://github.com/1jamesthompson1/AIML589), which
+investigates value alignment of LLMs with New Zealand population
+distributions from the World Values Survey.
+### Out-of-scope
+This model has not been safety-tuned for general-purpose deployment.
+It should not be used in production systems, for making decisions about
+people, or in contexts where reliability and safety are critical.
+## Limitations and biases
+- Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
+- The training data reflects the values of those who responded to the
+  survey and may not represent all New Zealanders.
+- LoRA adapters are subject to the limitations and biases of the base
+  model ([Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)).

adapter_config.json ADDED Viewed

	@@ -0,0 +1,50 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:60bbbe955503eda1da31b438dd00af3674448236c1dc1944aba4e0fe2be0ab9f
+size 116425880

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,154 @@

+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

environment.json ADDED Viewed

	@@ -0,0 +1,82 @@

+{
+  "Jinja2": "3.1.6",
+  "MarkupSafe": "3.0.3",
+  "PyYAML": "6.0.3",
+  "Pygments": "2.20.0",
+  "accelerate": "1.14.0",
+  "aiohappyeyeballs": "2.7.1",
+  "aiohttp": "3.14.3",
+  "aiosignal": "1.4.0",
+  "annotated-doc": "0.0.5",
+  "anyio": "4.14.2",
+  "attrs": "26.1.0",
+  "bitsandbytes": "0.50.0",
+  "certifi": "2026.7.22",
+  "charset-normalizer": "3.4.9",
+  "click": "8.4.2",
+  "cuda-bindings": "13.3.1",
+  "cuda-pathfinder": "1.6.0",
+  "cuda-toolkit": "13.0.3.0",
+  "datasets": "5.0.1",
+  "dill": "0.4.1",
+  "filelock": "3.32.0",
+  "frozenlist": "1.8.0",
+  "fsspec": "2026.6.0",
+  "h11": "0.16.0",
+  "hf-xet": "1.5.2",
+  "httpcore": "1.0.9",
+  "httpx": "0.28.1",
+  "huggingface_hub": "1.25.1",
+  "idna": "3.18",
+  "markdown-it-py": "4.2.0",
+  "mdurl": "0.1.2",
+  "mpmath": "1.3.0",
+  "multidict": "6.7.1",
+  "multiprocess": "0.70.19",
+  "networkx": "3.6.1",
+  "numpy": "2.5.1",
+  "nvidia-cublas": "13.1.1.3",
+  "nvidia-cuda-cupti": "13.0.85",
+  "nvidia-cuda-nvrtc": "13.0.88",
+  "nvidia-cuda-runtime": "13.0.96",
+  "nvidia-cudnn-cu13": "9.20.0.48",
+  "nvidia-cufft": "12.0.0.61",
+  "nvidia-cufile": "1.15.1.6",
+  "nvidia-curand": "10.4.0.35",
+  "nvidia-cusolver": "12.0.4.66",
+  "nvidia-cusparse": "12.6.3.3",
+  "nvidia-cusparselt-cu13": "0.8.1",
+  "nvidia-nccl-cu13": "2.29.7",
+  "nvidia-nvjitlink": "13.3.33",
+  "nvidia-nvshmem-cu13": "3.4.5",
+  "nvidia-nvtx": "13.0.85",
+  "packaging": "26.2",
+  "pandas": "3.0.5",
+  "peft": "0.20.0",
+  "pillow": "12.3.0",
+  "propcache": "0.5.2",
+  "psutil": "7.2.2",
+  "pyarrow": "25.0.0",
+  "python-dateutil": "2.9.0.post0",
+  "python-dotenv": "1.2.2",
+  "regex": "2026.7.19",
+  "requests": "2.34.2",
+  "rich": "15.0.0",
+  "safetensors": "0.8.0",
+  "setuptools": "83.0.0",
+  "shellingham": "1.5.4",
+  "six": "1.17.0",
+  "sympy": "1.14.0",
+  "tokenizers": "0.22.2",
+  "torch": "2.13.0",
+  "torchvision": "0.28.0",
+  "tqdm": "4.70.0",
+  "transformers": "5.14.1",
+  "triton": "3.7.1",
+  "trl": "1.9.2",
+  "typer": "0.27.0",
+  "typing_extensions": "4.16.0",
+  "urllib3": "2.7.0",
+  "xxhash": "3.8.1",
+  "yarl": "1.24.5"
+}

finetune_config.json ADDED Viewed

	@@ -0,0 +1,25 @@

+{
+  "model": "Qwen/Qwen3.5-9B",
+  "model_sha": "c202236235762e1c871ad0ccb60c8ee5ba337b9a",
+  "dataset": "sampled_response",
+  "dataset_sha": "38f42265e06d60afd0a5d2d45808202e60e95103",
+  "subpopulation": "cluster_0",
+  "gpu": "NVIDIA RTX 6000 Ada Generation",
+  "library_name": "peft",
+  "pipeline_tag": "text-generation",
+  "timestamp": "2026-08-05T15:57:46.834571",
+  "hyperparameters": {
+    "lora_r": 16,
+    "lora_alpha": 32,
+    "lora_dropout": 0.05,
+    "dora": false,
+    "quantization": null,
+    "lr": 0.0002,
+    "num_epochs": 3,
+    "batch_size": 4,
+    "gradient_accumulation_steps": 4,
+    "max_seq_length": 1024,
+    "warmup_ratio": 0.1,
+    "dtype": "bf16"
+  }
+}

processor_config.json ADDED Viewed

	@@ -0,0 +1,60 @@

+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "merge_size": 2,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 16777216,
+      "shortest_edge": 65536
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 2,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 25165824,
+      "shortest_edge": 4096
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,32 @@

+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

training.log ADDED Viewed

	@@ -0,0 +1,16 @@

+logging to output/sampled_response/cluster_0/training.log
+[sft] loading tokenizer...
+[sft] loading model...
+[sft] tokenizing 1206 examples...
+[sft] tokenizing validation split for eval...
+[data] loaded 150 validation examples (config=sampled_response, subpop=cluster_0)
+[sft] eval split: 150 examples
+[sft] starting training...
+[sft] saving...
+    saved to output/sampled_response/cluster_0
+    gpu: NVIDIA RTX 6000 Ada Generation  time: 1103s
+    training log -> output/sampled_response/cluster_0/training_log.json
+    run metadata -> output/sampled_response/cluster_0/finetune_config.json
+    environment -> output/sampled_response/cluster_0/environment.json
+    README -> output/sampled_response/cluster_0/README.md
+[upload] uploading output/sampled_response/cluster_0 to https://huggingface.co/1jamesthompson1/Qwen3.5-9B-nz-wvs-sampled_response-cluster_0...

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6c72d5679628010873e66334a077e6049a84120b54a54c16156c4987e4e2628e
+size 5713

training_log.json ADDED Viewed

	@@ -0,0 +1,484 @@

+[
+  {
+    "loss": 4.099612808227539,
+    "grad_norm": 3.3916518688201904,
+    "learning_rate": 7.82608695652174e-05,
+    "entropy": 1.0828558593988418,
+    "mean_token_accuracy": 0.44957960266619923,
+    "num_tokens": 24658.0,
+    "epoch": 0.13245033112582782,
+    "step": 10
+  },
+  {
+    "eval_loss": 0.5499903559684753,
+    "eval_runtime": 9.3737,
+    "eval_samples_per_second": 16.002,
+    "eval_steps_per_second": 4.054,
+    "eval_entropy": 0.5737218025483584,
+    "eval_mean_token_accuracy": 0.7952850923726433,
+    "eval_num_tokens": 24658.0,
+    "epoch": 0.13245033112582782,
+    "step": 10
+  },
+  {
+    "loss": 0.4331226825714111,
+    "grad_norm": 3.621782064437866,
+    "learning_rate": 0.00016521739130434784,
+    "entropy": 0.4598923083394766,
+    "mean_token_accuracy": 0.8161924198269844,
+    "num_tokens": 48994.0,
+    "epoch": 0.26490066225165565,
+    "step": 20
+  },
+  {
+    "eval_loss": 0.5726626515388489,
+    "eval_runtime": 9.3836,
+    "eval_samples_per_second": 15.985,
+    "eval_steps_per_second": 4.05,
+    "eval_entropy": 0.3104277225701432,
+    "eval_mean_token_accuracy": 0.8032894793309664,
+    "eval_num_tokens": 48994.0,
+    "epoch": 0.26490066225165565,
+    "step": 20
+  },
+  {
+    "loss": 0.37956321239471436,
+    "grad_norm": 3.6681301593780518,
+    "learning_rate": 0.00019414634146341464,
+    "entropy": 0.35528335012495516,
+    "mean_token_accuracy": 0.8529162302613258,
+    "num_tokens": 73216.0,
+    "epoch": 0.3973509933774834,
+    "step": 30
+  },
+  {
+    "eval_loss": 0.4902452528476715,
+    "eval_runtime": 9.6515,
+    "eval_samples_per_second": 15.542,
+    "eval_steps_per_second": 3.937,
+    "eval_entropy": 0.3591178828164151,
+    "eval_mean_token_accuracy": 0.8244517586733165,
+    "eval_num_tokens": 73216.0,
+    "epoch": 0.3973509933774834,
+    "step": 30
+  },
+  {
+    "loss": 0.3535017728805542,
+    "grad_norm": 1.529909372329712,
+    "learning_rate": 0.00018439024390243903,
+    "entropy": 0.31258961334824564,
+    "mean_token_accuracy": 0.8600528433918952,
+    "num_tokens": 97165.0,
+    "epoch": 0.5298013245033113,
+    "step": 40
+  },
+  {
+    "eval_loss": 0.5773032903671265,
+    "eval_runtime": 8.8699,
+    "eval_samples_per_second": 16.911,
+    "eval_steps_per_second": 4.284,
+    "eval_entropy": 0.40491867006609317,
+    "eval_mean_token_accuracy": 0.7638627878929439,
+    "eval_num_tokens": 97165.0,
+    "epoch": 0.5298013245033113,
+    "step": 40
+  },
+  {
+    "loss": 0.29483020305633545,
+    "grad_norm": 1.4113677740097046,
+    "learning_rate": 0.00017463414634146342,
+    "entropy": 0.3084935720078647,
+    "mean_token_accuracy": 0.8778989180922508,
+    "num_tokens": 121376.0,
+    "epoch": 0.6622516556291391,
+    "step": 50
+  },
+  {
+    "eval_loss": 0.5696188807487488,
+    "eval_runtime": 8.921,
+    "eval_samples_per_second": 16.814,
+    "eval_steps_per_second": 4.26,
+    "eval_entropy": 0.29807096403582317,
+    "eval_mean_token_accuracy": 0.8039630376978925,
+    "eval_num_tokens": 121376.0,
+    "epoch": 0.6622516556291391,
+    "step": 50
+  },
+  {
+    "loss": 0.26492722034454347,
+    "grad_norm": 2.0661537647247314,
+    "learning_rate": 0.00016487804878048782,
+    "entropy": 0.268452774733305,
+    "mean_token_accuracy": 0.8974904179573059,
+    "num_tokens": 145557.0,
+    "epoch": 0.7947019867549668,
+    "step": 60
+  },
+  {
+    "eval_loss": 0.8559898138046265,
+    "eval_runtime": 8.9149,
+    "eval_samples_per_second": 16.826,
+    "eval_steps_per_second": 4.263,
+    "eval_entropy": 0.2022110644458352,
+    "eval_mean_token_accuracy": 0.8000626611082178,
+    "eval_num_tokens": 145557.0,
+    "epoch": 0.7947019867549668,
+    "step": 60
+  },
+  {
+    "loss": 0.18335200548171998,
+    "grad_norm": 3.771090269088745,
+    "learning_rate": 0.0001551219512195122,
+    "entropy": 0.12369259365368634,
+    "mean_token_accuracy": 0.9281293749809265,
+    "num_tokens": 170605.0,
+    "epoch": 0.9271523178807947,
+    "step": 70
+  },
+  {
+    "eval_loss": 0.8269464373588562,
+    "eval_runtime": 8.912,
+    "eval_samples_per_second": 16.831,
+    "eval_steps_per_second": 4.264,
+    "eval_entropy": 0.23079313014290834,
+    "eval_mean_token_accuracy": 0.7807487523869464,
+    "eval_num_tokens": 170605.0,
+    "epoch": 0.9271523178807947,
+    "step": 70
+  },
+  {
+    "loss": 0.18180986642837524,
+    "grad_norm": 2.6767499446868896,
+    "learning_rate": 0.0001453658536585366,
+    "entropy": 0.16967114948324466,
+    "mean_token_accuracy": 0.9279846925484506,
+    "num_tokens": 193719.0,
+    "epoch": 1.0529801324503312,
+    "step": 80
+  },
+  {
+    "eval_loss": 0.8880820870399475,
+    "eval_runtime": 8.8086,
+    "eval_samples_per_second": 17.029,
+    "eval_steps_per_second": 4.314,
+    "eval_entropy": 0.2405679833119441,
+    "eval_mean_token_accuracy": 0.7817669220660862,
+    "eval_num_tokens": 193719.0,
+    "epoch": 1.0529801324503312,
+    "step": 80
+  },
+  {
+    "loss": 0.11491576433181763,
+    "grad_norm": 0.69949871301651,
+    "learning_rate": 0.000135609756097561,
+    "entropy": 0.14066557474434377,
+    "mean_token_accuracy": 0.9485534891486168,
+    "num_tokens": 218542.0,
+    "epoch": 1.185430463576159,
+    "step": 90
+  },
+  {
+    "eval_loss": 0.7975587844848633,
+    "eval_runtime": 8.8612,
+    "eval_samples_per_second": 16.928,
+    "eval_steps_per_second": 4.288,
+    "eval_entropy": 0.19119705641826026,
+    "eval_mean_token_accuracy": 0.7988721861651069,
+    "eval_num_tokens": 218542.0,
+    "epoch": 1.185430463576159,
+    "step": 90
+  },
+  {
+    "loss": 0.08289253115653991,
+    "grad_norm": 2.014707565307617,
+    "learning_rate": 0.00012585365853658536,
+    "entropy": 0.05604892138508148,
+    "mean_token_accuracy": 0.9736187428236007,
+    "num_tokens": 243302.0,
+    "epoch": 1.3178807947019868,
+    "step": 100
+  },
+  {
+    "eval_loss": 1.1870805025100708,
+    "eval_runtime": 8.8416,
+    "eval_samples_per_second": 16.965,
+    "eval_steps_per_second": 4.298,
+    "eval_entropy": 0.114802976807512,
+    "eval_mean_token_accuracy": 0.8206453684129213,
+    "eval_num_tokens": 243302.0,
+    "epoch": 1.3178807947019868,
+    "step": 100
+  },
+  {
+    "loss": 0.06358585357666016,
+    "grad_norm": 1.5396313667297363,
+    "learning_rate": 0.00011609756097560975,
+    "entropy": 0.04575981950329151,
+    "mean_token_accuracy": 0.9785756185650826,
+    "num_tokens": 267729.0,
+    "epoch": 1.4503311258278146,
+    "step": 110
+  },
+  {
+    "eval_loss": 0.8791955709457397,
+    "eval_runtime": 8.8863,
+    "eval_samples_per_second": 16.88,
+    "eval_steps_per_second": 4.276,
+    "eval_entropy": 0.17355985742515737,
+    "eval_mean_token_accuracy": 0.8070958703756332,
+    "eval_num_tokens": 267729.0,
+    "epoch": 1.4503311258278146,
+    "step": 110
+  },
+  {
+    "loss": 0.049860316514968875,
+    "grad_norm": 0.678949773311615,
+    "learning_rate": 0.00010634146341463416,
+    "entropy": 0.07773116882308387,
+    "mean_token_accuracy": 0.9833469331264496,
+    "num_tokens": 292279.0,
+    "epoch": 1.5827814569536423,
+    "step": 120
+  },
+  {
+    "eval_loss": 1.1025115251541138,
+    "eval_runtime": 8.9449,
+    "eval_samples_per_second": 16.769,
+    "eval_steps_per_second": 4.248,
+    "eval_entropy": 0.1594012568683339,
+    "eval_mean_token_accuracy": 0.7980889779956717,
+    "eval_num_tokens": 292279.0,
+    "epoch": 1.5827814569536423,
+    "step": 120
+  },
+  {
+    "loss": 0.07376678586006165,
+    "grad_norm": 2.6976561546325684,
+    "learning_rate": 9.658536585365854e-05,
+    "entropy": 0.0441209072858328,
+    "mean_token_accuracy": 0.9765854984521866,
+    "num_tokens": 316471.0,
+    "epoch": 1.7152317880794703,
+    "step": 130
+  },
+  {
+    "eval_loss": 1.2516794204711914,
+    "eval_runtime": 8.9032,
+    "eval_samples_per_second": 16.848,
+    "eval_steps_per_second": 4.268,
+    "eval_entropy": 0.12382908019782535,
+    "eval_mean_token_accuracy": 0.770441735261365,
+    "eval_num_tokens": 316471.0,
+    "epoch": 1.7152317880794703,
+    "step": 130
+  },
+  {
+    "loss": 0.02282872200012207,
+    "grad_norm": 0.8916032314300537,
+    "learning_rate": 8.682926829268293e-05,
+    "entropy": 0.0408805181941716,
+    "mean_token_accuracy": 0.9929233521223069,
+    "num_tokens": 340856.0,
+    "epoch": 1.847682119205298,
+    "step": 140
+  },
+  {
+    "eval_loss": 1.273831844329834,
+    "eval_runtime": 8.873,
+    "eval_samples_per_second": 16.905,
+    "eval_steps_per_second": 4.283,
+    "eval_entropy": 0.11421615798501175,
+    "eval_mean_token_accuracy": 0.7660557698262366,
+    "eval_num_tokens": 340856.0,
+    "epoch": 1.847682119205298,
+    "step": 140
+  },
+  {
+    "loss": 0.020523886382579803,
+    "grad_norm": 2.7703590393066406,
+    "learning_rate": 7.707317073170732e-05,
+    "entropy": 0.028159727145248326,
+    "mean_token_accuracy": 0.9912698417901993,
+    "num_tokens": 364928.0,
+    "epoch": 1.980132450331126,
+    "step": 150
+  },
+  {
+    "eval_loss": 1.5218627452850342,
+    "eval_runtime": 8.8524,
+    "eval_samples_per_second": 16.945,
+    "eval_steps_per_second": 4.293,
+    "eval_entropy": 0.09264942740800937,
+    "eval_mean_token_accuracy": 0.7557487519163835,
+    "eval_num_tokens": 364928.0,
+    "epoch": 1.980132450331126,
+    "step": 150
+  },
+  {
+    "loss": 0.015184570848941804,
+    "grad_norm": 0.04658397287130356,
+    "learning_rate": 6.731707317073171e-05,
+    "entropy": 0.016722216987737307,
+    "mean_token_accuracy": 0.9981203000796469,
+    "num_tokens": 387571.0,
+    "epoch": 2.1059602649006623,
+    "step": 160
+  },
+  {
+    "eval_loss": 1.6605678796768188,
+    "eval_runtime": 8.8568,
+    "eval_samples_per_second": 16.936,
+    "eval_steps_per_second": 4.29,
+    "eval_entropy": 0.07781053681920687,
+    "eval_mean_token_accuracy": 0.770441735261365,
+    "eval_num_tokens": 387571.0,
+    "epoch": 2.1059602649006623,
+    "step": 160
+  },
+  {
+    "loss": 0.006011705845594406,
+    "grad_norm": 0.013418244197964668,
+    "learning_rate": 5.756097560975609e-05,
+    "entropy": 0.007514824125610176,
+    "mean_token_accuracy": 0.9968627452850342,
+    "num_tokens": 412476.0,
+    "epoch": 2.23841059602649,
+    "step": 170
+  },
+  {
+    "eval_loss": 1.763867974281311,
+    "eval_runtime": 8.8357,
+    "eval_samples_per_second": 16.977,
+    "eval_steps_per_second": 4.301,
+    "eval_entropy": 0.07184799665475639,
+    "eval_mean_token_accuracy": 0.7737312089455756,
+    "eval_num_tokens": 412476.0,
+    "epoch": 2.23841059602649,
+    "step": 170
+  },
+  {
+    "loss": 0.004858419671654702,
+    "grad_norm": 0.14898061752319336,
+    "learning_rate": 4.7804878048780485e-05,
+    "entropy": 0.004944811350287637,
+    "mean_token_accuracy": 0.9980769231915474,
+    "num_tokens": 436969.0,
+    "epoch": 2.370860927152318,
+    "step": 180
+  },
+  {
+    "eval_loss": 1.827174425125122,
+    "eval_runtime": 8.8372,
+    "eval_samples_per_second": 16.974,
+    "eval_steps_per_second": 4.3,
+    "eval_entropy": 0.06969891959229962,
+    "eval_mean_token_accuracy": 0.7720864721034703,
+    "eval_num_tokens": 436969.0,
+    "epoch": 2.370860927152318,
+    "step": 180
+  },
+  {
+    "loss": 0.005421416833996773,
+    "grad_norm": 0.08094702661037445,
+    "learning_rate": 3.804878048780488e-05,
+    "entropy": 0.004511333805567119,
+    "mean_token_accuracy": 0.9966666668653488,
+    "num_tokens": 461400.0,
+    "epoch": 2.5033112582781456,
+    "step": 190
+  },
+  {
+    "eval_loss": 1.859080195426941,
+    "eval_runtime": 8.9117,
+    "eval_samples_per_second": 16.832,
+    "eval_steps_per_second": 4.264,
+    "eval_entropy": 0.06829966303944514,
+    "eval_mean_token_accuracy": 0.7753759457876808,
+    "eval_num_tokens": 461400.0,
+    "epoch": 2.5033112582781456,
+    "step": 190
+  },
+  {
+    "loss": 0.005886765569448471,
+    "grad_norm": 0.007371403276920319,
+    "learning_rate": 2.8292682926829267e-05,
+    "entropy": 0.006164468544739066,
+    "mean_token_accuracy": 0.9951255336403847,
+    "num_tokens": 485443.0,
+    "epoch": 2.6357615894039736,
+    "step": 200
+  },
+  {
+    "eval_loss": 1.8880680799484253,
+    "eval_runtime": 8.8924,
+    "eval_samples_per_second": 16.868,
+    "eval_steps_per_second": 4.273,
+    "eval_entropy": 0.06679329828732175,
+    "eval_mean_token_accuracy": 0.7753759457876808,
+    "eval_num_tokens": 485443.0,
+    "epoch": 2.6357615894039736,
+    "step": 200
+  },
+  {
+    "loss": 0.0009710748679935932,
+    "grad_norm": 0.0006506519857794046,
+    "learning_rate": 1.853658536585366e-05,
+    "entropy": 0.002073692464909982,
+    "mean_token_accuracy": 1.0,
+    "num_tokens": 509990.0,
+    "epoch": 2.7682119205298013,
+    "step": 210
+  },
+  {
+    "eval_loss": 1.9050503969192505,
+    "eval_runtime": 8.8469,
+    "eval_samples_per_second": 16.955,
+    "eval_steps_per_second": 4.295,
+    "eval_entropy": 0.06654788765100468,
+    "eval_mean_token_accuracy": 0.7737312089455756,
+    "eval_num_tokens": 509990.0,
+    "epoch": 2.7682119205298013,
+    "step": 210
+  },
+  {
+    "loss": 0.01160859763622284,
+    "grad_norm": 0.004122971091419458,
+    "learning_rate": 8.780487804878048e-06,
+    "entropy": 0.00959118189784931,
+    "mean_token_accuracy": 0.9908364906907081,
+    "num_tokens": 534335.0,
+    "epoch": 2.9006622516556293,
+    "step": 220
+  },
+  {
+    "eval_loss": 1.9109818935394287,
+    "eval_runtime": 8.8226,
+    "eval_samples_per_second": 17.002,
+    "eval_steps_per_second": 4.307,
+    "eval_entropy": 0.06608304418458781,
+    "eval_mean_token_accuracy": 0.7737312089455756,
+    "eval_num_tokens": 534335.0,
+    "epoch": 2.9006622516556293,
+    "step": 220
+  },
+  {
+    "eval_loss": 1.9071061611175537,
+    "eval_runtime": 8.9501,
+    "eval_samples_per_second": 16.76,
+    "eval_steps_per_second": 4.246,
+    "eval_entropy": 0.06653332771269274,
+    "eval_mean_token_accuracy": 0.7737312089455756,
+    "eval_num_tokens": 552465.0,
+    "epoch": 3.0,
+    "step": 228
+  },
+  {
+    "train_runtime": 1102.1217,
+    "train_samples_per_second": 3.283,
+    "train_steps_per_second": 0.207,
+    "total_flos": 3.178328110793933e+16,
+    "train_loss": 0.2925478722695915,
+    "epoch": 3.0,
+    "step": 228
+  }
+]