| language: | |
| - en | |
| license: apache-2.0 | |
| tags: | |
| - glove | |
| - lora | |
| - distillation | |
| - bpe | |
| - cl100k_base | |
| base_model: jsanzolac/bpe_glove_512 | |
| datasets: | |
| - jsanzolac/qwen3_emb_512 | |
| - jsanzolac/qwen3_emb_512_packed | |
| # bpe_glove_512_lora_v1 | |
| LoRA drifts on top of frozen `jsanzolac/bpe_glove_512` BPE-GloVe-512 embeddings, distilled | |
| from `Qwen/Qwen3-Embedding-8B` (MRL-truncated to 512 dims). | |
| **Variant 1 loss:** `位_c路InfoNCE + 位_D路||蟻_T - 蟻_S||_F虏` with `位_c=1.0`, `位_D=0.1`. | |
| Each `rank_<r>/` folder contains: | |
| - `checkpoint_final.pt` | |
| - `config.json` | |
| - `vectors_drifted.txt` | |
| - `train_log.jsonl` | |
| Ranks shipped: [512, 256, 128, 64, 32, 16, 8, 4, 2] | |