Weather LLM โ€” SFT (run1 stage2, step 2500)

Supervised fine-tuning checkpoint: run1_stage2/checkpoint-2500. This repository contains both the published model artifacts and supporting project code files.

Model Details

  • Model type: Causal language model for text generation (transformers).
  • Training flow: weather-domain pretraining followed by supervised fine-tuning (SFT).
  • Checkpoint lineage: this release corresponds to the uploaded run1_stage2/checkpoint-2500 model.
  • Tokenizer: Hub-compatible tokenizer files are included in this repository.
  • Context length: configured by uploaded tokenizer/model files (model_max_length=2048).

Data Sources

  • NOAA-derived historical weather data processed into natural-language records.
  • Optional support text from Wikipedia used during corpus building.
  • Synthetic and grounded instruction-response examples used for SFT dataset construction.

Code in Files Section

Code under src/, scripts/, and configs/ in the HF Files section is used for:

  • data building and preprocessing,
  • tokenizer training,
  • model pretraining and SFT training workflows,
  • reproducibility of training/evaluation steps.

Intended Use

  • Instruction-following weather-analysis style generation.
  • Educational and research usage for domain-focused LLM workflows.

Out-of-Scope / Limitations

  • Not for real-time weather operations or safety-critical decision-making.
  • Outputs may be incorrect, incomplete, or outdated and should be independently validated.
  • Not a replacement for authoritative meteorological services.

Inference

There are two inference entrypoints, and they are intentionally different:

  1. Hugging Face Hub inference (loads by model repo ID).
  2. Project-local inference script (loads by local model directory path used during local training workflow).

Hugging Face Hub Snippet

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "AuraWorxAI/weather-llm-initial"
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(repo)

Requires transformers and sentencepiece.

Project-Local Inference Snippet

python -m weather_llm.inference.generate \
  --model_dir artifacts/checkpoints/sft/run1_stage2 \
  --prompt "Compare summer weather patterns in Arizona vs Washington."

The HF snippet loads from a Hub model ID, while the local snippet loads from a filesystem path produced in local training runs.

Contact

Maintainer: AuraWorxAI

Downloads last month
3
Safetensors
Model size
1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support