qwen3-embedding-0.6b-swedish-superlim

596M Qwen3 embedding model that beats KBLab/sentence-bert-swedish-cased on both SuperLim tasks that card published.

It is a weight soup of two Swedish fine-tunes of Qwen/Qwen3-Embedding-0.6B:

0.15 × CoSENT (STS) + 0.85 × CoSENT-then-FAQ

One file, one forward. Not an ensemble at inference. Use the full 1024 dimensions — unlike the BGE sister model, a 768-d cut loses the dual win.

A stronger Swedish ship on the same protocol is oxfrug/bge-m3-swedish-superlim (568M, 0.835 / 0.651). This repo is the Qwen-family 0.6B.

Test (SuperLim-2, held-out)

Model	SweParaphrase ρ	SweFAQ
KBLab v2.0 (our re-run)	0.8207	0.5780
Qwen3-Embedding-0.6B zero-shot	0.7294	0.5229
CoSENT parent	0.8279	0.5229
CoSENT→FAQ parent	0.8205	0.5963
this soup (0.15 / 0.85)	0.8226	0.6055

SweParaphrase test was never in train. SweFAQ used official train only. Dev also clears (0.848 / 0.645).

MTEB(Scandinavian, v1)

Same five models, mteb 2.19.3, max_seq_length=512. 27/28 tasks (DKHateClassification gated, skipped). SweFAQ here is nDCG@10 and is supervised for this soup. This is a SuperLim-targeted checkpoint — it does not beat KBLab on the Swedish-only MTEB mean.

Swedish tasks:

Task	KBLab	Qwen 0.6B	this soup	BGE-sv-mix@768
DalajClassification	0.5003	0.5018	0.5036	0.4993
SweRecClassification	0.6858	0.6532	0.7300	0.7647
SwedishSentimentClassification	0.8737	0.8411	0.9170	0.9293
SweFaqRetrieval nDCG@10 †	0.7331	0.6747	0.7295	0.8221
SwednRetrieval	0.7067	0.6638	0.6517	0.7809
SwednClusteringP2P	0.3641	0.3298	0.3520	0.3554
SwednClusteringS2S	0.2486	0.0716	0.0893	0.1991
MassiveIntent (sv)	0.6587	0.5772	0.5968	0.6824
MassiveScenario (sv)	0.7475	0.6543	0.6737	0.7553
SV mean (9)	0.6132	0.5519	0.5826	0.6432

† Supervised for this soup. SwednRetrieval has no train overlap and went the wrong way vs both KBLab and vanilla Qwen.

Nordic category means (27 tasks):

	KBLab	Qwen 0.6B	this soup	BGE-sv-mix@768
Classification (12)	0.5243	0.5503	0.5707	0.5960
Retrieval (7)	0.4466	0.6117	0.6066	0.6915
Clustering (6)	0.4091	0.3618	0.3741	0.4101
Bitext (2)	0.4338	0.7314	0.6846	0.6876
Overall (27)	0.4719	0.5377	0.5448	0.5862

The soup slightly lifts vanilla Qwen on the Nordic mean. It still trails KBLab on the Swedish subset. Use the BGE sister model if you want the broader Swedish retrieval lift.

Raw task JSON: mteb/MTEB_Scandinavian_v1/. SuperLim test dump: superlim/test.json. These files document the run; they do not put the model on the official MTEB leaderboard.

Use

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("oxfrug/qwen3-embedding-0.6b-swedish-superlim")

# STS / clustering: no prompt
a, b = model.encode(
    ["Ett plan lyfter.", "Ett flygplan lyfter."],
    normalize_embeddings=True,
)

# Retrieval: official Qwen query prompt on the question only
q = model.encode(
    ["När får jag föräldrapenning?"],
    prompt_name="query",
    normalize_embeddings=True,
)
d = model.encode(
    ["Du kan ansöka på Mina sidor."],
    normalize_embeddings=True,
)

Keep all 1024 dims. @768 is 0.821 / 0.578 — FAQ only ties KBLab.

How it was trained

1. CoSENT on all graded SweParaphrase-train pairs (label / 5). SuperLim dev/test blocked. This is the first 0.6B Qwen in this lab over 0.821 Spearman.
2. Light MNRL continue on SweFAQ-train + healthcare Q–A (Qwen query prefix). FAQ recovered; STS fell to 0.8205.
3. Soup the two state_dicts. Mix weights that clear both bars on dev also clear test.

Do not 4-bit this checkpoint if you want the STS claim. The lead is +0.002.

Limits

• Fine-tune + soup, not a new architecture.
• SweFAQ n=109; this is +3 items vs the KBLab re-run.
• Behind the BGE Swedish mix on both tasks.
• SuperLim claim is SweParaphrase + SweFAQ only. On MTEB Scandinavian the Swedish-only mean is still under KBLab.

License

Apache 2.0, same as Qwen3-Embedding-0.6B. Fine-tune by oxfrug.