anjieliu/baseline-guzheng

Guzheng baseline — from-scratch hierarchical patch+char GPT

A patch+character hierarchical GPT trained from scratch on guzheng ABC. Same architecture as NotaGen-medium (231M params), but no pre-training. Its sole purpose is the comparison against the fine-tuned NotaGen — if this baseline performs significantly worse on the same data, the gap measures the contribution of NotaGen's pre-training corpus, not architecture or recipe.

This is not a recommended generation model. Use the fine-tuned NotaGen for actual guzheng generation.

Model

Spec	Value
Architecture	Patch-level GPT (16 layers) + Char-level GPT (3 layers)
Hidden size	1024
Total parameters	~231M
Patch size	16 chars
Max patches per sequence	256
Vocab	128 (ASCII)
Pre-training	none — random initialization

Training

• Data: same 125 pieces as the NotaGen fine-tune (bundled in data/dataset.jsonl).
• Sampling: identical triangular ±3-semitone key augmentation + 5× repertoire oversampling.
• Optimizer: AdamW, peak LR 3e-4 with 200-step warmup, cosine decay over 200 epochs.
• Hardware: Apple MPS, fp32, batch size 1, gradient accumulation 1.
• Best epoch: 89, eval loss 1.04.

The eval loss gap to fine-tuned NotaGen (0.47) is ~2.2× — most of the quality difference attributable to pre-training in the experimental setup.

Usage

import torch
from safetensors.torch import load_file
# Code lives in the GitHub repo:
# github.com/anjieliu/guzheng/blob/main/baseline/train.py
from train import Baseline

state = load_file("model.safetensors")
model = Baseline(d_model=1024, n_heads=16, patch_layers=16,
                 char_layers=3, d_ff=4096, max_patches=256, dropout=0.0)
model.load_state_dict(state)
model.eval()

Full training/inference code: github.com/anjieliu/guzheng (baseline/generate.py is a working entry point).

Citation

If you use this baseline (e.g., to reproduce the comparison), please cite the thesis.

License

Code: MIT (see GitHub repo). Weights and dataset: CC-BY-SA 4.0.