Instructions to use nazdef/1gpu-llm-medium-en-it-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nazdef/1gpu-llm-medium-en-it-base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nazdef/1gpu-llm-medium-en-it-base")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nazdef/1gpu-llm-medium-en-it-base")
model = AutoModelForCausalLM.from_pretrained("nazdef/1gpu-llm-medium-en-it-base")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nazdef/1gpu-llm-medium-en-it-base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nazdef/1gpu-llm-medium-en-it-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nazdef/1gpu-llm-medium-en-it-base

SGLang

How to use nazdef/1gpu-llm-medium-en-it-base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nazdef/1gpu-llm-medium-en-it-base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nazdef/1gpu-llm-medium-en-it-base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nazdef/1gpu-llm-medium-en-it-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nazdef/1gpu-llm-medium-en-it-base with Docker Model Runner:
```
docker model run hf.co/nazdef/1gpu-llm-medium-en-it-base
```

1gpu-llm-medium-en-it-base / training_config.yaml

nazdef

release: publish official 1gpu-llm medium base from step14700

52ee5b8 verified 7 days ago

Raw

History Blame Contribute Delete

2.91 kB

	# Replay-only decay tail continuation to recover the missing checkpoints after the original
	# step_13500 decay-only run ran out of disk during checkpoint serialization.
	# This resumes from the last valid saved checkpoint (`step_14200`) and reproduces the
	# remaining original cooldown trajectory through `step_14850`.
	#
	# Important:
	# - launch this replay with `--resume-mode model_only`
	# so the run keeps the model weights + global_step from `step_14200`
	# while intentionally resetting optimizer/scheduler to continue the missing tail
	# - the original decay tail started at `step_13500`
	# - original decay length was `1350` steps
	# - therefore the correct tail end is `step_14850`
	# - to continue the same inverse-proportional curve from `step_14200`, this replay
	# starts from the exact LR implied by the original tail at `step_14200`
	# and decays over the remaining `650` steps down to the same `final_lr`

	resume_from: /mnt/dati/llm-nanochat/checkpoints_archive/tier2_20260629_medium_prune/20260628_resume-gpt2medium-gpt2preln-k20-wsddecayonly-lr2e-4-anchor20k-final2e5-webwiki-step13500/step_14200.pt
	dataset_dir: /mnt/apps/llm-nanochat/datasets/202605141153_fineweb50_wiki50_50en_50it_score100_2500context_5Btokens_tok_20260515_en50it50_webwiki_stratified_500M
	output_dir: /mnt/apps/llm-nanochat/artifacts/runs/20260629_resume-gpt2medium-gpt2preln-k20-wsddecayonly-rerunmissing-lr3p5294e5-anchor20k-final2e5-webwiki-step14200-to14850
	tokenizer_dir: /mnt/apps/llm-nanochat/tokenizers/tokenizer_20260515_en50it50_webwiki_stratified_500M
	seed: 1337

	model:
	architecture: gpt2
	block_type: gpt2_prelayernorm
	tie_word_embeddings: true
	vocab_size: 32000
	dim: 1024
	n_layers: 24
	n_heads: 16

	training:
	sequence_length: 2500
	max_steps: 14850
	batch_size: 2
	grad_accum_steps: 48

	learning_rate: 3.529411764705883e-05
	peak_lr: 3.529411764705883e-05
	lr_schedule: wsd-decay-only

	warmup_steps: 0
	stable_steps: 0
	decay_steps: 650
	final_lr: 2.0e-05

	adamw_betas:
	- 0.9
	- 0.95
	adamw_eps: 1.0e-08
	weight_decay: 0.1
	clip_grad_norm: 1.0

	save_every_steps: 50
	checkpoint_dir: /mnt/apps/llm-nanochat/checkpoints/20260629_resume-gpt2medium-gpt2preln-k20-wsddecayonly-rerunmissing-lr3p5294e5-anchor20k-final2e5-webwiki-step14200-to14850
	precision: bf16

	evaluation:
	validation_every_steps: 50
	validation_max_batches: 128
	probe_every_steps: 1000
	probe_tokenizer_dir: /mnt/apps/llm-nanochat/tokenizers/tokenizer_20260515_en50it50_webwiki_stratified_500M
	probe_max_new_tokens: 32
	probe_prompts:
	en:
	- prompt: "The capital of Italy is"
	expected_next_text: " Rome"
	- prompt: "A small language model should"
	expected_next_text: " be"
	it:
	- prompt: "La capitale d'Italia è"
	expected_next_text: " Roma"
	- prompt: "Un piccolo modello linguistico dovrebbe"
	expected_next_text: " essere"