---
language:
- en
license: apache-2.0
tags:
- glove
- lora
- distillation
- bpe
- cl100k_base
base_model: jsanzolac/bpe_glove_512
datasets:
- jsanzolac/qwen3_emb_512
- jsanzolac/qwen3_emb_512_packed
---

# bpe_glove_512_lora_v1

LoRA drifts on top of frozen `jsanzolac/bpe_glove_512` BPE-GloVe-512 embeddings, distilled
from `Qwen/Qwen3-Embedding-8B` (MRL-truncated to 512 dims).

**Variant 1 loss:** `λ_c·InfoNCE + λ_D·||ρ_T - ρ_S||_F²` with `λ_c=1.0`, `λ_D=0.1`.

Each `rank_<r>/` folder contains:
- `checkpoint_final.pt`
- `config.json`
- `vectors_drifted.txt`
- `train_log.jsonl`

Ranks shipped: [512, 256, 128, 64, 32, 16, 8, 4, 2]