Released fine-tuned QuizGen models
Collection
Most current fine-tuned and tested models for Quizgen downstream task from Rockship Co. • 4 items • Updated
How to use from
Unsloth StudioInstall Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for vietphuon/Llama-3.2-1B-Instruct-bnb-4bit-quizgen-241025-1 to start chattingUsing HuggingFace Spaces for Unsloth
# No setup required# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for vietphuon/Llama-3.2-1B-Instruct-bnb-4bit-quizgen-241025-1 to start chattingLoad model with FastModel
pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
model_name="vietphuon/Llama-3.2-1B-Instruct-bnb-4bit-quizgen-241025-1",
max_seq_length=2048,
)Quick Links
DATASET
- What's new?: Use the version 3.2 of dataset (Langfuse + AWS) that has better quality:
- Remove all the 10, 15 question count, just focus on 5 question count
- Fix all the Vietnamese quiz (make sure the output is Vietnamese)
- Fix some lazy duplicated topic (Biglead, Computing)
- Remove Paragraph, replace Paragraph with MCQ for all data points
- Train using the default training config (60 step, linear lr)
TRAINING
- 1075.8979 seconds used for training.
- 17.93 minutes used for training.
- Peak reserved memory = 7.877 GB.
- Peak reserved memory for training = 6.729 GB.
- Peak reserved memory % of max memory = 53.411 %.
- Peak reserved memory for training % of max memory = 45.627 %.
- Final loss = 0.740000
- View full training here: https://wandb.ai/vietphuongnguyen2602-rockship/huggingface/runs/04u9obeu
FINAL BENCHMARKING
- Time to First Token (TTFT): 0.002s
- Time Per Output Token (TPOT): 40.85ms/token
- Throughput (token/s): 25.66token/s
- Average Token Latency (ms/token): 40.90ms/token
- Total Generation Time: 63.015s
- Input Tokenization Time: 0.008s
- Input Tokens: 1909
- Output Tokens: 984
- Total Tokens: 2892
- Memory Usage (GPU): 1.49GB
Uploaded model
- Developed by: vietphuon
- License: apache-2.0
- Finetuned from model : unsloth/Llama-3.2-1B-Instruct-bnb-4bit
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for vietphuon/Llama-3.2-1B-Instruct-bnb-4bit-quizgen-241025-1
Base model
meta-llama/Llama-3.2-1B-Instruct Quantized
unsloth/Llama-3.2-1B-Instruct-bnb-4bit
Install Unsloth Studio (macOS, Linux, WSL)
# Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for vietphuon/Llama-3.2-1B-Instruct-bnb-4bit-quizgen-241025-1 to start chatting