KonradBRG commited on
Commit
e7d58f4
·
verified ·
1 Parent(s): de09e5e

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +13 -4
README.md CHANGED
@@ -14,14 +14,21 @@ tags:
14
 
15
  # glot500-base-plains-cree-en-clkd-direct
16
 
17
- Glot500-base adapted for Plains Cree figurative language detection via CLKD, without TLM warmup.
18
 
19
  ## Method
20
 
21
- CLKD applied directly to the pretrained [Glot500-base](https://huggingface.co/cis-lmu/glot500-base) checkpoint (no TLM warmup).
22
- Tests whether Glot500's broad multilingual representations are sufficient for cross-lingual figurative language transfer into Plains Cree without explicit alignment fine-tuning.
23
 
24
- **Teacher:** `KonradBRG/deberta-v3-base-figurative`
 
 
 
 
 
 
 
 
25
  **Temperature:** 2.0 | **Epochs:** 10 | **Hardware:** 1× NVIDIA A100 40 GB
26
 
27
  **Labels:** `literal` (0), `idiom` (1), `metaphor` (2), `simile` (3)
@@ -31,8 +38,10 @@ Tests whether Glot500's broad multilingual representations are sufficient for cr
31
 
32
  If you use this model, please cite the associated thesis/paper (TBD).
33
 
 
34
  ## Data
35
 
36
  Training data includes:
37
  - [Bloomfield (1934) *Plains Cree Texts*](https://bloomfield.kiyanaw.net) (scraped and sentence-aligned)
38
  - [EdTeKLA Indigenous Languages Corpora](https://github.com/EdTeKLA/IndigenousLanguages_Corpora)
 
 
14
 
15
  # glot500-base-plains-cree-en-clkd-direct
16
 
17
+ cis-lmu/glot500-base adapted for Plains Cree figurative language detection via Cross-Lingual Knowledge Distillation (CLKD) (no TLM warmup).
18
 
19
  ## Method
20
 
21
+ Inspired by [Cross-Lingual Knowledge Distillation (ACL 2023)](https://aclanthology.org/2023.findings-acl.885/).
 
22
 
23
+ **CLKD pipeline:**
24
+ 1. Teacher: `KonradBRG/deberta-v3-base-figurative` (frozen, English)
25
+ 2. Student base: `cis-lmu/glot500-base`
26
+ 3. All layers trainable.
27
+
28
+ The teacher predicts soft label distributions on English translations of Plains Cree sentences.
29
+ The student is trained to match these distributions on the Cree side via KL divergence.
30
+
31
+ **Training data:** ~9,000 Cree–English parallel sentence pairs
32
  **Temperature:** 2.0 | **Epochs:** 10 | **Hardware:** 1× NVIDIA A100 40 GB
33
 
34
  **Labels:** `literal` (0), `idiom` (1), `metaphor` (2), `simile` (3)
 
38
 
39
  If you use this model, please cite the associated thesis/paper (TBD).
40
 
41
+
42
  ## Data
43
 
44
  Training data includes:
45
  - [Bloomfield (1934) *Plains Cree Texts*](https://bloomfield.kiyanaw.net) (scraped and sentence-aligned)
46
  - [EdTeKLA Indigenous Languages Corpora](https://github.com/EdTeKLA/IndigenousLanguages_Corpora)
47
+