Plains Cree CLKD Models
Collection
Figurative language classifiers distilled into Plains Cree via CLKD. DeBERTa-v3 teacher, three student architectures, two warmup strategies. • 7 items • Updated
glot500-base-plains-cree-en-clkd-direct
cis-lmu/glot500-base adapted for Plains Cree figurative language detection via Cross-Lingual Knowledge Distillation (CLKD) (no TLM warmup).
Method
Inspired by Cross-Lingual Knowledge Distillation (ACL 2023).
CLKD pipeline:
- Teacher:
KonradBRG/deberta-v3-base-figurative(frozen, English) - Student base:
cis-lmu/glot500-base - All layers trainable.
The teacher predicts soft label distributions on English translations of Plains Cree sentences. The student is trained to match these distributions on the Cree side via KL divergence.
Training data: ~9,000 Cree–English parallel sentence pairs Temperature: 2.0 | Epochs: 10 | Hardware: 1× NVIDIA A100 40 GB
Labels: literal (0), idiom (1), metaphor (2), simile (3)
Citation
If you use this model, please cite the associated thesis/paper (TBD).
Data
Training data includes:
- Bloomfield (1934) Plains Cree Texts (scraped and sentence-aligned)
- EdTeKLA Indigenous Languages Corpora
- Downloads last month
- 127
Model tree for KonradBRG/glot500-base-plains-cree-en-clkd-direct
Base model
cis-lmu/glot500-base