File size: 1,320 Bytes
786c17c 986f3b5 786c17c 986f3b5 786c17c 986f3b5 786c17c 986f3b5 786c17c 986f3b5 786c17c 986f3b5 786c17c 986f3b5 786c17c 986f3b5 786c17c 986f3b5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | ---
base_model:
- Qwen/Qwen3-4B
tags:
- text-generation-inference
- transformers
- trl
- qwen
- wsd
- ambiguity
license: apache-2.0
language:
- en
datasets:
- deshanksuman/Reasoning_WSD_dataset
pipeline_tag: text-classification
---
# Uploaded model
- **Developed by:** deshanksuman
- **License:** apache-2.0
- **Finetuned from model :** Qwen/Qwen3-4B
# Dataset
Fews Training data arranged in the format of Instruction, Input and output with advanced Reasonining for sense identification.
The data generation has been semi automated using the Arcee models.
The data has been validated by the human for it's structure and the correctnes.
The data source can be accessed here:
deshanksuman/Reasoning_WSD_dataset
# Hyperparameter for Training
- per_device_train_batch_size=4,
- gradient_accumulation_steps=8,
- warmup_steps=50,
- num_train_epochs=2,
- learning_rate=2e-4,
- fp16=not torch.cuda.is_bf16_supported(),
- bf16=torch.cuda.is_bf16_supported(),
- logging_steps=10,
- optim="adamw_torch",
- weight_decay=0.01,
- lr_scheduler_type="linear",
- seed=3407
This is developed by Deshan Sumanathilaka https://sumanathilaka.github.io
# Acknowledgement
We acknowledge the support of the Supercomputing Wales project, which is part-funded by the European Regional Development Fund (ERDF) via Welsh Government. |