Instructions to use tuandunghcmut/Qwen3-8B-Private with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tuandunghcmut/Qwen3-8B-Private with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base_model, "tuandunghcmut/Qwen3-8B-Private")

Transformers

How to use tuandunghcmut/Qwen3-8B-Private with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tuandunghcmut/Qwen3-8B-Private")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tuandunghcmut/Qwen3-8B-Private")
model = AutoModelForCausalLM.from_pretrained("tuandunghcmut/Qwen3-8B-Private")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tuandunghcmut/Qwen3-8B-Private with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tuandunghcmut/Qwen3-8B-Private"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tuandunghcmut/Qwen3-8B-Private",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tuandunghcmut/Qwen3-8B-Private

SGLang

How to use tuandunghcmut/Qwen3-8B-Private with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tuandunghcmut/Qwen3-8B-Private" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tuandunghcmut/Qwen3-8B-Private",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tuandunghcmut/Qwen3-8B-Private" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tuandunghcmut/Qwen3-8B-Private",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tuandunghcmut/Qwen3-8B-Private with Docker Model Runner:
```
docker model run hf.co/tuandunghcmut/Qwen3-8B-Private
```

See axolotl config

axolotl version: 0.13.0.dev0

base_model: Qwen/Qwen3-8B
# Automatically upload checkpoint and final model to HF
# hub_model_id: username/custom_model_name
hub_model_id: tuandunghcmut/Qwen3-8B-Private


plugins:
  - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
strict: false



chat_template: qwen3
datasets:
  # - path: trendmicro-ailab/Primus-Seed
  #   type: chat_template
    # split: train[:20%]
    # split: train
    # field_messages: conversations
    # message_property_mappings:
    #   role: from
    #   content: value
  - path: trendmicro-ailab/Primus-Reasoning
    type: chat_template
    # split: train[:20%]
    split: train
    split_thinking: true
    chat_template: qwen3
    field_messages: messages
    message_property_mappings:
      role: role
      content: content

      
val_set_size: 0.075
output_dir: ./outputs/out2
dataset_prepared_path: last_run_prepared

# sequence_len: 2048
sequence_len: 3072
sample_packing: true
eval_sample_packing: true


load_in_4bit: true
adapter: qlora
# lora_r: 16
# lora_alpha: 32

# lora_r: 32
# lora_alpha: 64


lora_r: 64
lora_alpha: 128

lora_target_modules:
  - q_proj
  - k_proj
  - v_proj
  - o_proj
  - down_proj
  - up_proj
lora_mlp_kernel: true
lora_qkv_kernel: true
lora_o_kernel: true

wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:

gradient_accumulation_steps: 2
micro_batch_size: 4
num_epochs: 30
optimizer: adamw_torch_4bit
lr_scheduler: cosine
learning_rate: 0.00002

bf16: auto
tf32: true

gradient_checkpointing: offload
gradient_checkpointing_kwargs:
  use_reentrant: false
resume_from_checkpoint:
logging_steps: 1
flash_attention: true

warmup_ratio: 0.1
evals_per_epoch: 4
saves_per_epoch: 1
weight_decay: 0.01
special_tokens:

# save_first_step: true  # uncomment this to validate checkpoint saving works with your config

Qwen3-8B-Private

This model is a fine-tuned version of Qwen/Qwen3-8B on the trendmicro-ailab/Primus-Reasoning dataset. It achieves the following results on the evaluation set:

Loss: 0.9940
Memory/max Active (gib): 8.07
Memory/max Allocated (gib): 8.07
Memory/device Reserved (gib): 10.78

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Use OptimizerNames.ADAMW_TORCH_4BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 471
training_steps: 4710

Training results

Training Loss	Epoch	Step	Validation Loss	Active (gib)	Allocated (gib)	Reserved (gib)
No log	0	0	1.2742	7.9	7.9	10.87
1.2102	0.2540	40	1.2336	8.07	8.07	10.76
1.0179	0.5079	80	1.0396	8.07	8.07	10.78
1.0278	0.7619	120	0.9600	8.07	8.07	10.78
0.9719	1.0127	160	0.9087	8.07	8.07	10.76
0.9052	1.2667	200	0.8665	8.07	8.07	10.76
0.8255	1.5206	240	0.8274	8.07	8.07	10.76
0.813	1.7746	280	0.7970	8.07	8.07	10.76
0.819	2.0254	320	0.7757	8.07	8.07	10.76
0.8138	2.2794	360	0.7570	8.07	8.07	10.78
0.7745	2.5333	400	0.7433	8.07	8.07	10.78
0.7587	2.7873	440	0.7317	8.07	8.07	10.78
0.7222	3.0381	480	0.7224	8.07	8.07	10.78
0.7092	3.2921	520	0.7127	8.07	8.07	10.76
0.6615	3.5460	560	0.7071	8.07	8.07	10.76
0.715	3.8	600	0.7026	8.07	8.07	10.76
0.6747	4.0508	640	0.6995	8.07	8.07	10.78
0.7012	4.3048	680	0.6939	8.07	8.07	10.78
0.698	4.5587	720	0.6921	8.07	8.07	10.78
0.6591	4.8127	760	0.6874	8.07	8.07	10.78
0.6716	5.0635	800	0.6854	8.07	8.07	10.78
0.7077	5.3175	840	0.6858	8.07	8.07	10.78
0.6817	5.5714	880	0.6822	8.07	8.07	10.78
0.6668	5.8254	920	0.6800	8.07	8.07	10.78
0.6543	6.0762	960	0.6819	8.07	8.07	10.78
0.6378	6.3302	1000	0.6798	8.07	8.07	10.78
0.5922	6.5841	1040	0.6779	8.07	8.07	10.78
0.637	6.8381	1080	0.6773	8.07	8.07	10.78
0.6478	7.0889	1120	0.6768	8.07	8.07	10.78
0.6429	7.3429	1160	0.6781	8.07	8.07	10.76
0.5847	7.5968	1200	0.6777	8.07	8.07	10.76
0.6423	7.8508	1240	0.6734	8.07	8.07	10.76
0.5793	8.1016	1280	0.6788	8.07	8.07	10.76
0.5706	8.3556	1320	0.6802	8.07	8.07	10.78
0.5729	8.6095	1360	0.6770	8.07	8.07	10.78
0.6757	8.8635	1400	0.6755	8.07	8.07	10.78
0.5643	9.1143	1440	0.6806	8.07	8.07	10.78
0.5391	9.3683	1480	0.6825	8.07	8.07	10.76
0.5565	9.6222	1520	0.6829	8.07	8.07	10.76
0.5931	9.8762	1560	0.6777	8.07	8.07	10.76
0.5608	10.1270	1600	0.6863	8.07	8.07	10.78
0.5635	10.3810	1640	0.6864	8.07	8.07	10.78
0.5379	10.6349	1680	0.6835	8.07	8.07	10.78
0.5436	10.8889	1720	0.6858	8.07	8.07	10.78
0.5511	11.1397	1760	0.6944	8.07	8.07	10.78
0.536	11.3937	1800	0.6952	8.07	8.07	10.76
0.5371	11.6476	1840	0.6952	8.07	8.07	10.76
0.5763	11.9016	1880	0.6910	8.07	8.07	10.76
0.5802	12.1524	1920	0.7053	8.07	8.07	10.78
0.5802	12.4063	1960	0.7062	8.07	8.07	10.78
0.5422	12.6603	2000	0.7028	8.07	8.07	10.78
0.478	12.9143	2040	0.7027	8.07	8.07	10.78
0.5467	13.1651	2080	0.7207	8.07	8.07	10.78
0.5345	13.4190	2120	0.7182	8.07	8.07	10.76
0.4922	13.6730	2160	0.7169	8.07	8.07	10.76
0.5062	13.9270	2200	0.7165	8.07	8.07	10.76
0.4797	14.1778	2240	0.7369	8.07	8.07	10.76
0.4438	14.4317	2280	0.7335	8.07	8.07	10.78
0.4726	14.6857	2320	0.7293	8.07	8.07	10.78
0.4651	14.9397	2360	0.7305	8.07	8.07	10.78
0.4489	15.1905	2400	0.7580	8.07	8.07	10.78
0.4447	15.4444	2440	0.7494	8.07	8.07	10.78
0.5027	15.6984	2480	0.7481	8.07	8.07	10.78
0.4883	15.9524	2520	0.7504	8.07	8.07	10.78
0.4223	16.2032	2560	0.7677	8.07	8.07	10.78
0.492	16.4571	2600	0.7688	8.07	8.07	10.78
0.4541	16.7111	2640	0.7730	8.07	8.07	10.78
0.4801	16.9651	2680	0.7688	8.07	8.07	10.78
0.3932	17.2159	2720	0.7981	8.07	8.07	10.78
0.4209	17.4698	2760	0.7900	8.07	8.07	10.76
0.3891	17.7238	2800	0.7938	8.07	8.07	10.76
0.4155	17.9778	2840	0.7903	8.07	8.07	10.76
0.347	18.2286	2880	0.8233	8.07	8.07	10.78
0.3558	18.4825	2920	0.8172	8.07	8.07	10.78
0.4365	18.7365	2960	0.8230	8.07	8.07	10.78
0.4451	18.9905	3000	0.8181	8.07	8.07	10.78
0.3627	19.2413	3040	0.8568	8.07	8.07	10.78
0.337	19.4952	3080	0.8403	8.07	8.07	10.78
0.4094	19.7492	3120	0.8426	8.07	8.07	10.78
0.4225	20.0	3160	0.8332	8.07	8.07	10.78
0.3481	20.2540	3200	0.8752	8.07	8.07	10.74
0.3947	20.5079	3240	0.8700	8.07	8.07	10.78
0.4106	20.7619	3280	0.8649	8.07	8.07	10.76
0.3333	21.0127	3320	0.8730	8.07	8.07	10.78
0.3558	21.2667	3360	0.8966	8.07	8.07	10.78
0.3625	21.5206	3400	0.8912	8.07	8.07	10.78
0.3429	21.7746	3440	0.8918	8.07	8.07	10.78
0.3597	22.0254	3480	0.9114	8.07	8.07	10.78
0.3445	22.2794	3520	0.9217	8.07	8.07	10.78
0.3366	22.5333	3560	0.9176	8.07	8.07	10.78
0.3557	22.7873	3600	0.9181	8.07	8.07	10.78
0.3937	23.0381	3640	0.9393	8.07	8.07	10.78
0.3161	23.2921	3680	0.9391	8.07	8.07	10.78
0.3272	23.5460	3720	0.9413	8.07	8.07	10.78
0.3755	23.8	3760	0.9378	8.07	8.07	10.78
0.2966	24.0508	3800	0.9564	8.07	8.07	10.78
0.2639	24.3048	3840	0.9591	8.07	8.07	10.78
0.306	24.5587	3880	0.9599	8.07	8.07	10.78
0.3215	24.8127	3920	0.9581	8.07	8.07	10.78
0.392	25.0635	3960	0.9713	8.07	8.07	10.78
0.3494	25.3175	4000	0.9794	8.07	8.07	10.78
0.3245	25.5714	4040	0.9720	8.07	8.07	10.78
0.3053	25.8254	4080	0.9724	8.07	8.07	10.78
0.304	26.0762	4120	0.9849	8.07	8.07	10.78
0.3043	26.3302	4160	0.9843	8.07	8.07	10.76
0.3426	26.5841	4200	0.9846	8.07	8.07	10.76
0.2979	26.8381	4240	0.9853	8.07	8.07	10.76
0.3526	27.0889	4280	0.9912	8.07	8.07	10.78
0.3095	27.3429	4320	0.9885	8.07	8.07	10.78
0.2983	27.5968	4360	0.9898	8.07	8.07	10.78
0.3086	27.8508	4400	0.9912	8.07	8.07	10.78
0.3194	28.1016	4440	0.9919	8.07	8.07	10.78
0.2484	28.3556	4480	0.9947	8.07	8.07	10.78
0.3458	28.6095	4520	0.9943	8.07	8.07	10.78
0.3467	28.8635	4560	0.9945	8.07	8.07	10.78
0.3295	29.1143	4600	0.9936	8.07	8.07	10.78
0.3555	29.3683	4640	0.9942	8.07	8.07	10.78
0.3273	29.6222	4680	0.9940	8.07	8.07	10.78

Framework versions

PEFT 0.17.1
Transformers 4.56.1
Pytorch 2.7.1+cu126
Datasets 4.0.0
Tokenizers 0.22.1

Downloads last month: 8

Model tree for tuandunghcmut/Qwen3-8B-Private

Base model

Qwen/Qwen3-8B-Base

Finetuned

Qwen/Qwen3-8B

Adapter

(1828)

this model

tuandunghcmut
/

Qwen3-8B-Private

Qwen3-8B-Private

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Model tree for tuandunghcmut/Qwen3-8B-Private

Dataset used to train tuandunghcmut/Qwen3-8B-Private