luciayen commited on
Commit
efa8d0f
·
verified ·
1 Parent(s): c8a25b7

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +123 -0
README.md ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - sw
4
+ tags:
5
+ - sign-language
6
+ - pose-landmarks
7
+ - mediapipe
8
+ - transformer
9
+ - african-sign-language
10
+ - KSL
11
+ - isolated-sign-recognition
12
+ license: apache-2.0
13
+ datasets:
14
+ - luciayen/KSL-Hand-Landmarks
15
+ metrics:
16
+ - accuracy
17
+ model-index:
18
+ - name: AfriSignEncoder Exp1 — KSL Baseline
19
+ results:
20
+ - task:
21
+ type: video-classification
22
+ name: Isolated Sign Word Recognition
23
+ dataset:
24
+ name: KSL-Hand-Landmarks
25
+ type: luciayen/KSL-Hand-Landmarks
26
+ metrics:
27
+ - type: accuracy
28
+ value: 1.0
29
+ name: Validation Accuracy
30
+ ---
31
+
32
+ # AfriSignEncoder Exp1 — KSL Baseline (LandmarkTransformer)
33
+
34
+ Part of the **AfriSignEncoder** research project: a multilingual African sign language recognition benchmark.
35
+ This checkpoint is the **Experiment 1 single-language baseline** for Kenyan Sign Language (KSL).
36
+
37
+ > **Scope note:** This baseline covers only 4 KSL glosses (father, hello, is, my) from a small
38
+ > demo-scale dataset. The 100% accuracy reflects the simplicity of the 4-class task and should
39
+ > not be generalised to full KSL vocabulary recognition.
40
+
41
+ ## Model Description
42
+
43
+ Same LandmarkTransformer architecture as the CASL baseline, with a 4-class head.
44
+
45
+ | Component | Value |
46
+ |---|---|
47
+ | Architecture | LandmarkTransformer (custom) |
48
+ | Input | (B, 64, 225) float32 — 75 keypoints × 3 coords per frame |
49
+ | Embedding dim | 256 |
50
+ | Attention heads | 8 |
51
+ | Encoder layers | 4 |
52
+ | Feed-forward dim | 1,024 |
53
+ | Positional encoding | Learned |
54
+ | Classification head | Linear 256 → 4 |
55
+ | Parameters | ~3.24 M |
56
+
57
+ ## Dataset
58
+
59
+ **KSL Hand Landmarks** — 4 Kenyan Sign Language words.
60
+
61
+ | Split | Samples | Classes |
62
+ |---|---|---|
63
+ | Train | 694 | 4 |
64
+ | Test | 124 | 4 |
65
+
66
+ Source: Kaggle `joanwachuka/ksl-hand-landmarks` → parquet at `luciayen/KSL-Hand-Landmarks`.
67
+
68
+ **Landmark caveat:** The original `.npy` files contain MediaPipe **Hands**-only keypoints
69
+ (42 joints × 3 = 126D). These are placed in dimensions [0:126]; dimensions [126:225]
70
+ (the 99 pose body dims) are zero-padded. The model therefore learns from hand shape
71
+ and motion only; the padded dimensions are always zero and contribute no signal.
72
+
73
+ **Data leakage fix:** The Kaggle archive contains two sub-directories — `data_split/`
74
+ (official train/val/test) and `dataset3/` (pre-split source). All 124 test samples appear
75
+ verbatim in `dataset3/`. The upload script excludes `dataset3/` entirely, giving a clean
76
+ train=694 / test=124 split with zero overlap.
77
+
78
+ ## Training
79
+
80
+ Identical protocol to the CASL baseline.
81
+
82
+ | Setting | Value |
83
+ |---|---|
84
+ | Optimiser | AdamW (lr=3e-4, wd=1e-4) |
85
+ | LR schedule | OneCycleLR cosine |
86
+ | Max epochs | 60 |
87
+ | Batch size | 64 |
88
+ | Loss | CrossEntropy + label_smoothing=0.1 |
89
+ | Early stopping | patience=12 on val acc |
90
+ | Normalisation | Per-feature z-score (stats stored in checkpoint) |
91
+
92
+ ## Results
93
+
94
+ | Metric | Value |
95
+ |---|---|
96
+ | Best validation accuracy | **100%** |
97
+ | Best checkpoint epoch | 3 |
98
+ | Final epoch (early stop) | 15 |
99
+ | Number of classes | 4 |
100
+
101
+ The 100% result is expected: 4 highly phonologically distinct signs, ~173 training samples
102
+ per class, and a well-regularised model. This result validates the pipeline; it does not
103
+ benchmark KSL at meaningful scale.
104
+
105
+ ## Checkpoint Contents
106
+
107
+ ```python
108
+ import torch
109
+ ck = torch.load("pytorch_model.bin", map_location="cpu")
110
+ # Keys: epoch, val_acc, model (state_dict), l2i (label→index dict),
111
+ # mean (tensor 225,), std (tensor 225,)
112
+ # l2i = {"father": 0, "hello": 1, "is": 2, "my": 3}
113
+ ```
114
+
115
+ ## Limitations
116
+
117
+ - Only 4 classes — not a meaningful KSL benchmark.
118
+ - Pose dims are always zero (hand-only source data). Re-extraction with MediaPipe Holistic is planned.
119
+ - Small dataset; results will change with more data.
120
+
121
+ ## Citation / Project
122
+
123
+ AfriSignEncoder research project, CMU, 2026. GitHub: `africansl_encoder`.