xlm-mlm-100-1280-plains-cree-en-clkd-frozen12

FacebookAI/xlm-mlm-100-1280 adapted for Plains Cree figurative language detection via Cross-Lingual Knowledge Distillation (CLKD).

Method

Inspired by Cross-Lingual Knowledge Distillation (ACL 2023).

CLKD pipeline:

1. Teacher: KonradBRG/deberta-v3-base-figurative (frozen, English)
2. Student base: KonradBRG/xlm-mlm-100-1280-plains-cree-en-tlm
3. Layers 0–11 frozen to preserve cross-lingual alignment from TLM; remaining layers + classification head trained.

The teacher predicts soft label distributions on English translations of Plains Cree sentences. The student is trained to match these distributions on the Cree side via KL divergence.

Training data: ~9,000 Cree–English parallel sentence pairs Temperature: 2.0 | Epochs: 10 | Hardware: 1× NVIDIA A100 40 GB

Labels: literal (0), idiom (1), metaphor (2), simile (3)

Citation

If you use this model, please cite the associated thesis/paper (TBD).

Data

Training data includes:

• Bloomfield (1934) Plains Cree Texts (scraped and sentence-aligned)
• EdTeKLA Indigenous Languages Corpora