---
language: []
license: apache-2.0
tags:
  - audio
  - music
  - source-separation
  - guitar
  - demucs
  - htdemucs
  - pytorch
datasets:
  - MoisesDB
metrics:
  - sdr
library_name: pytorch
model-index:
  - name: htdemucs-6s-guitar-ft
    results:
      - task:
          type: audio-source-separation
          name: Music Source Separation
        dataset:
          name: MoisesDB v0.1
          type: moises/moisesdb
        metrics:
          - type: sdr
            value: 8.73
            name: Guitar SDR (dB)
          - type: sir
            value: 21.30
            name: Guitar SIR (dB)
          - type: sar
            value: 6.65
            name: Guitar SAR (dB)
---

# Guitar Domain Expert HTDemucs 6s

A domain-specialized fine-tune of Meta AI's **htdemucs_6s** trained exclusively to extract clean, isolated guitar stems from dense, real-world studio mixes.

> **TL;DR:** Drop in any song. Get back a clean guitar stem. Works on clean riffs, distorted rock, acoustic, and layered studio productions.

---

## Results

Evaluated on a 24-track held-out split of **MoisesDB v0.1** using full-track BSS metrics (deterministic `shifts=0`):

| Model | SDR ↑ (dB) | SIR ↑ (dB) | SAR ↑ (dB) |
|---|---|---|---|
| Stock `htdemucs_6s` (Meta AI baseline) | 8.42 | 20.41 | 6.28 |
| **`htdemucs-6s-guitar-ft` (this model)** | **8.73** | **21.30** | **6.65** |
| Δ improvement | **+0.31** | **+0.89** | **+0.37** |

The **+0.89 dB SIR gain** is the headline result — it directly measures guitar clarity and bleed rejection. Less drum transient, less bass rumble, less cymbal wash bleeding into your extracted guitar stem.

---

## Quick Start

```python
from demucs.pretrained import get_model
from demucs.apply import apply_model
from huggingface_hub import hf_hub_download
import torch
import soundfile as sf

GUITAR_STEM_INDEX = 4  # htdemucs_6s: [drums, bass, other, vocals, guitar, piano]

# Download fine-tuned weights from HuggingFace
checkpoint_path = hf_hub_download(
    repo_id="adityalakhani/htdemucs-6s-guitar-ft", 
    filename="guitar_htdemucs_6s.pt"
)

# Load base model architecture
model = get_model("htdemucs_6s")

# Load and unwrap fine-tuned weights (training dict)
checkpoint = torch.load(checkpoint_path, map_location="cpu", weights_only=True)
state_dict = checkpoint["model_state_dict"]

# Remap keys to match demucs' BagOfModels wrapper
state_dict = {f"models.0.{k}": v for k, v in state_dict.items()}
model.load_state_dict(state_dict)
model.eval()

# Load audio (must be 44100 Hz. soundfile avoids torchaudio Windows DLL issues)
audio_np, sr = sf.read("your_song.wav", always_2d=True, dtype="float32")
waveform = torch.from_numpy(audio_np.T)  # (C, T)
if waveform.shape[0] == 1:
    waveform = waveform.repeat(2, 1)

# Separate using apply_model for proper overlap-add chunking
with torch.no_grad():
    stems = apply_model(model, waveform.unsqueeze(0), shifts=1, overlap=0.25)
    guitar = stems[0, GUITAR_STEM_INDEX]

sf.write("guitar_stem.wav", guitar.T.numpy(), samplerate=44100, subtype="PCM_24")
print("Done.")
```

Or use the included CLI script for any audio file format (mp3, flac, wav, etc.):

```bash
python inference.py --input song.mp3 --output guitar_stem.wav
```

---

## Model Details

### Architecture

This model is a fine-tuned checkpoint of **htdemucs_6s** — Meta AI's Hybrid Transformer Demucs (6-source variant). The architecture processes audio simultaneously in two domains:

- **Time domain:** Raw waveform → 1D Conv encoder → Cross-Domain Transformer → 1D ConvTranspose decoder
- **Frequency domain:** STFT spectrogram → 2D Conv encoder → Cross-Domain Transformer → 2D ConvTranspose → iSTFT

Both decoder outputs are summed to produce each separated stem. The model outputs 6 stems: `drums`, `bass`, `other`, `vocals`, `guitar` (index 4), `piano`.

**~80M total parameters.**

### What makes this different from stock `htdemucs_6s`?

The pre-trained `htdemucs_6s` is a general-purpose separator trained on MUSDB18-HQ — a broad mix of musical genres. This fine-tuned version was subsequently trained on **MoisesDB** (240 real multitrack songs with professionally isolated guitar stems), which produces a model that has developed a much stronger prior for real guitar acoustic and spectral characteristics.

The result is a model that:
- Rejects drum transient bleed more aggressively (+0.89 dB SIR)
- More precisely follows guitar sustain curves vs. background noise
- Is better calibrated for the full guitar frequency range (clean highs to distorted lows)

---

## Training Details

### Dataset

**MoisesDB v0.1** — 240 real commercial multitrack songs with professionally isolated stems:
- Each song contains ground-truth isolated guitar WAV files
- Covers Rock, Singer-Songwriter, Pop, Latin, and Electronic genres
- Guitar stems are harmonically and rhythmically interlocked with their actual mix — no synthetic mix construction

**Split:** 216 songs training / 24 songs validation (stratified by genre via fixed-seed shuffle).

### Fine-Tuning Strategy

Training was split into two phases to prevent catastrophic forgetting:

| Phase | Layers Active | LR | Epochs |
|---|---|---|---|
| Phase 1 (Warmup) | Decoders only (Encoder + Transformer frozen) | 3×10⁻⁴ | 20 |
| Phase 2 (Fine-tune) | Full network unfrozen | 1×10⁻⁴ | ~55 (early stopped) |

The GradScaler was explicitly reset at the Phase 1 → Phase 2 transition to prevent the sudden encoder gradient regime from destabilizing the decoder weights learned in Phase 1.

### Loss Function

```
L_total = L1(pred, target)
        + 0.5 × MultiResolutionSTFT(pred, target)
        + SI-SDR(pred, target)    # weighted to zero on silent chunks (RMS < 1e-4)
```

- **L1** provides waveform fidelity
- **Multi-Resolution STFT** adds spectral phase and magnitude matching across 3 FFT sizes
- **SI-SDR** (Scale-Invariant SDR) directly optimizes the deployment metric — added in V4 to break through the +5.3 dB plateau

### Data Augmentation

- **Remix augmentation (80%):** For each training chunk, non-guitar stems (drums, bass, vocals, etc.) are independently swapped from random other songs in the dataset. This prevents the model from memorising song-specific mix signatures.
- **Guitar gain randomisation:** Guitar stem scaled randomly in `[-6 dB, +6 dB]` before mixing.
- **Chunk length:** 7.8 seconds (matches the model's effective receptive field).

### Hyperparameters

| Parameter | Value |
|---|---|
| Batch size | 1 |
| Gradient accumulation | 4 steps (effective batch = 4) |
| Mixed precision | fp16 (torch.autocast + GradScaler) |
| Early stopping patience | 10 epochs (on val loss) |
| Gradient clipping | max_norm = 1.0 |
| Total training epochs | 75 (early stopped) |

---

## Limitations

- **Heavy multi-guitar polyphony:** When a mix contains 3+ overlapping guitars (lead + rhythm + acoustic), the model extracts all of them into a single stem — it does not separate individual guitar layers from each other. This is by design for the source separation stage of a larger pipeline.
- **Extreme distortion + cymbal bleed:** Very high-gain metal tones (wall-of-sound production) with heavy cymbal wash remain the hardest cases. The model reduces bleed significantly but does not eliminate it.
- **Piano / keyboard ambiguity:** Synth pads in the upper mid-frequency range can occasionally bleed into the guitar output on dense electronic productions. This is inherited from the base `htdemucs_6s` architecture.
- **Mono input:** Mono audio is automatically upmixed to stereo before inference; results may be slightly less precise than native stereo input.

---

## How to Cite

If you use this model in research or a project, please cite:

```bibtex
@misc{lakhani2026htdemucs_guitar_ft,
  title     = {Guitar Domain Expert HTDemucs 6s: A Domain-Specialized Fine-Tune of HTDemucs for Guitar Stem Extraction},
  author    = {Lakhani, Aditya},
  year      = {2026},
  url       = {https://huggingface.co/aditya-lakhani/htdemucs-6s-guitar-ft},
}
```

---

## Acknowledgements

- **[Meta AI Research](https://github.com/facebookresearch/demucs)** — HTDemucs architecture and pre-trained weights (`demucs==4.0.1`)
- **[Moises.ai](https://moises.ai/)** — MoisesDB v0.1 multitrack dataset used for domain fine-tuning
- **[auraloss](https://github.com/csteinmetz1/auraloss)** — Multi-Resolution STFT loss implementation (`auraloss==0.4.0`)

---

## License

This model is released under the **Apache 2.0** license. The base `htdemucs_6s` weights (Meta AI) are released under MIT. MoisesDB data is used in compliance with Moises.ai's research licensing terms; raw audio data is not redistributed.