swdq/acestep-v15-jpdenpa-ft

ACE-Step 1.5 turbo — full fine-tune (jpdenpa)

Fine-tuned decoder weights for ACE-Step/Ace-Step1.5 (turbo variant). This is a full fine-tune, not a LoRA: every decoder parameter was updated, so there is no adapter to attach — the weights replace the base decoder's.

Demo: swdq/acestep-jpdenpa

What is in the file

epoch_40.pt is a torch.save dict:

{"trainable": {param_name: bf16_tensor, ...},   # 476 tensors, 1,575,458,880 params
 "meta": {"unfreeze": "all", "epoch": 40, "loss": ..., "lr": 2e-5, "optimizer": "adafactor"}}

Parameter names are relative to model.decoder, so applying it is a copy:

import torch
from huggingface_hub import hf_hub_download

blob = torch.load(hf_hub_download("swdq/acestep-v15-jpdenpa-ft", "epoch_40.pt"),
                  map_location="cpu", weights_only=True)
own = dict(model.decoder.named_parameters())
with torch.no_grad():
    for k, t in blob["trainable"].items():
        p = own[k]
        p.data.copy_(t.to(dtype=p.dtype, device=p.device))

Training

Data	11 Japanese songs (anime rock, vocaloid, denpa pop, punk, alt-pop)
Trainable	1,575,458,880 params — the entire decoder, 65.8% of the model
Optimizer	Adafactor, lr 2e-5, cosine, batch 1 × grad accum 4
Precision	bf16 base weights, fp32 masters for the unfrozen parameters
Hardware	2 × RTX 5060 Ti 16 GB, decoder sharded across both cards
Checkpoint	epoch 40 of a run stopped at 41

Captions were trained with the tag jpdenpa prepended, so keeping that tag in the prompt is what pulls the fine-tuned style in.

Why fp32 masters: updating bf16 weights in place discards any update below ~2^-8 of the weight's magnitude. At |w| ≈ 0.02 the resolution is ~7.8e-5 while an lr 2e-5 step is ~1e-7, so almost every update would round to zero.

Why two GPUs: fp32 masters plus fp32 grads for the decoder come to ~19 GB, which does not fit on one 16 GB card. The decoder blocks are split across both with accelerate's align-device hooks — sequential model parallelism, not DDP.

Measurement

The flow-matching training loss is not a usable progress signal here: it redraws noise and the timestep every step, so consecutive epoch losses differ mostly by the draw. Over epochs 15–33 of this run the epoch-to-epoch jitter was 0.024 against a real per-epoch improvement of 0.0026 — nine times larger than the signal. Progress was tracked instead with a harness that pins the noise per sample and evaluates over a fixed timestep grid, so every checkpoint sees byte-identical inputs.