Hugging Face's logo

KonradBRG
/
xlm-mlm-100-1280-plains-cree-en-clkd-frozen12

Text Classification
Safetensors
Plains Cree
xlm
figurative-language
plains-cree
clkd
low-resource
Model card Files
xet
 Community
xlm-mlm-100-1280-plains-cree-en-clkd-frozen12 / README.md
KonradBRG's picture
KonradBRG
Upload README.md with huggingface_hub
14c0850 verified
preview code
|
Raw
History Blame Contribute Delete
1.5 kB
metadata 
base_model: FacebookAI/xlm-mlm-100-1280
language:
  - crk
license: cc-by-4.0
pipeline_tag: text-classification
tags:
  - figurative-language
  - text-classification
  - plains-cree
  - clkd
  - low-resource

xlm-mlm-100-1280-plains-cree-en-clkd-frozen12

FacebookAI/xlm-mlm-100-1280 adapted for Plains Cree figurative language detection via Cross-Lingual Knowledge Distillation (CLKD).

Method

Inspired by Cross-Lingual Knowledge Distillation (ACL 2023).

CLKD pipeline:

  1. Teacher: KonradBRG/deberta-v3-base-figurative (frozen, English)
  2. Student base: KonradBRG/xlm-mlm-100-1280-plains-cree-en-tlm
  3. Layers 0–11 frozen to preserve cross-lingual alignment from TLM; remaining layers + classification head trained.

The teacher predicts soft label distributions on English translations of Plains Cree sentences. The student is trained to match these distributions on the Cree side via KL divergence.

Training data: ~9,000 Cree–English parallel sentence pairs Temperature: 2.0 | Epochs: 10 | Hardware: 1× NVIDIA A100 40 GB

Labels: literal (0), idiom (1), metaphor (2), simile (3)

Citation

If you use this model, please cite the associated thesis/paper (TBD).

Data

Training data includes: