File size: 647 Bytes
ddcb77d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | ---
language:
- en
license: apache-2.0
tags:
- glove
- lora
- distillation
- bpe
- cl100k_base
base_model: jsanzolac/bpe_glove_512
datasets:
- jsanzolac/qwen3_emb_512
- jsanzolac/qwen3_emb_512_packed
---
# bpe_glove_512_lora_v1
LoRA drifts on top of frozen `jsanzolac/bpe_glove_512` BPE-GloVe-512 embeddings, distilled
from `Qwen/Qwen3-Embedding-8B` (MRL-truncated to 512 dims).
**Variant 1 loss:** `位_c路InfoNCE + 位_D路||蟻_T - 蟻_S||_F虏` with `位_c=1.0`, `位_D=0.1`.
Each `rank_<r>/` folder contains:
- `checkpoint_final.pt`
- `config.json`
- `vectors_drifted.txt`
- `train_log.jsonl`
Ranks shipped: [512, 256, 128, 64, 32, 16, 8, 4, 2]
|