| --- |
| base_model: |
| - Qwen/Qwen3-4B |
| tags: |
| - text-generation-inference |
| - transformers |
| - trl |
| - qwen |
| - wsd |
| - ambiguity |
| license: apache-2.0 |
| language: |
| - en |
| datasets: |
| - deshanksuman/Reasoning_WSD_dataset |
| pipeline_tag: text-classification |
| --- |
| |
| # Uploaded model |
|
|
| - **Developed by:** deshanksuman |
| - **License:** apache-2.0 |
| - **Finetuned from model :** Qwen/Qwen3-4B |
|
|
|
|
| # Dataset |
| Fews Training data arranged in the format of Instruction, Input and output with advanced Reasonining for sense identification. |
| The data generation has been semi automated using the Arcee models. |
| The data has been validated by the human for it's structure and the correctnes. |
|
|
| The data source can be accessed here: |
| deshanksuman/Reasoning_WSD_dataset |
|
|
|
|
| # Hyperparameter for Training |
|
|
|
|
| - per_device_train_batch_size=4, |
| - gradient_accumulation_steps=8, |
| - warmup_steps=50, |
| - num_train_epochs=2, |
| - learning_rate=2e-4, |
| - fp16=not torch.cuda.is_bf16_supported(), |
| - bf16=torch.cuda.is_bf16_supported(), |
| - logging_steps=10, |
| - optim="adamw_torch", |
| - weight_decay=0.01, |
| - lr_scheduler_type="linear", |
| - seed=3407 |
| |
| |
| |
| This is developed by Deshan Sumanathilaka https://sumanathilaka.github.io |
| |
| # Acknowledgement |
| We acknowledge the support of the Supercomputing Wales project, which is part-funded by the European Regional Development Fund (ERDF) via Welsh Government. |
