---
title: Sign Language Bridge
emoji: 🤟
colorFrom: indigo
colorTo: purple
sdk: static
app_file: index.html
short_description: ASL video to English translation (Qwen3-VL-2B LoRA)
license: apache-2.0
models:
  - mamounyosef/sign-language-bridge
  - Qwen/Qwen3-VL-2B-Instruct
datasets:
  - how2sign
  - openasl
tags:
  - sign-language
  - sign-language-translation
  - asl
  - video-text-to-text
---

# sign-language-bridge — ASL → English

> ### ⏳ Awaiting GPU hardware
>
> The Gradio app in this repo (`app.py`, `preprocessing.py`) is complete and
> written for ZeroGPU. It is **not running yet**: hosting a Gradio Space
> requires either PRO or a free-account ZeroGPU slot, and this account is
> currently under the 30-day age threshold for the free allowance.
>
> To switch it on, replace the frontmatter above with:
>
> ```yaml
> sdk: gradio
> sdk_version: 6.24.0
> app_file: app.py
> python_version: "3.12"
> startup_duration_timeout: 45m
> ```
>
> and attach `zero-a10g` hardware — either by subscribing to
> [PRO](https://huggingface.co/pro), waiting out the 30 days, or being awarded a
> community GPU grant. No code changes are needed.

Demo for [`mamounyosef/sign-language-bridge`](https://huggingface.co/mamounyosef/sign-language-bridge),
a multi-tier LoRA / RSLoRA fine-tune of
[`Qwen/Qwen3-VL-2B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct)
for continuous American Sign Language → English translation, trained on
How2Sign + OpenASL.

Upload or record a 1–15 second ASL clip and the Space returns an English
translation, plus the preprocessed clip the model actually saw.

## Why this Space reproduces the training preprocessing

The adapter was trained with three preprocessing stages applied to **every**
split, so inference has to reproduce them or the model receives an
out-of-distribution input and quality collapses. `preprocessing.py` ports them
from the [project repo](https://github.com/mamounyosef/sign-language-bridge):

| Stage | What it does | Ported from |
|---|---|---|
| Pose-guided signer crop | MediaPipe `PoseLandmarker`; union of upper-body + hand landmarks across the clip, padded 25 %, snapped to a multiple of 32 | `data_code/signer_cropper.py` |
| CLAHE | Contrast equalisation on the L channel in LAB (clip 2.0, 8×8 tiles) | `Qwen3VLCollator._apply_clahe_opencv` |
| Landmark overlay | RTMPose Wholebody; 6 upper-body joints + 21 keypoints per hand drawn as a 1 px skeleton | `data_code/21_extract_landmarks.py`, `Qwen3VLCollator._apply_landmark_overlay` |

Each stage has a UI toggle so you can see how much the model leans on it.

Video sampling and generation settings match the evaluated checkpoint
(step 4,610) exactly:

- 20 fps, `min_pixels` 4,096, `max_pixels` 184,320, `total_pixels` 20,971,520
- System prompt: *"You are a sign language translator."*
- User prompt: *"Translate this American Sign Language video into English."*
- Beam size 5, length penalty 0.6, no-repeat n-gram 4, repetition penalty 1.1, max 32 new tokens

## Limitations

This is a **research preview**. On the author's How2Sign test partition
(944 clips, a custom 90/5/5 split — *not* the official How2Sign split, so the
numbers are not comparable to published results):

| BLEU-1 | BLEU-4 | chrF | ROUGE-L | METEOR | WER |
|---|---|---|---|---|---|
| 19.76 | 1.64 | 17.42 | 10.43 | 9.71 | 112.5 % |

Output is fluent English in the register of the target captions and often
topically appropriate, but frequently disagrees with the reference word for
word — and can be confidently wrong. Do **not** use it where a mistranslation
could cause harm (medical, legal, safety-critical, or emergency settings).
The model has seen almost only How2Sign / OpenASL signers, framings, and
lighting, and generalises poorly outside them.

## License

Apache 2.0, matching the adapter and the `Qwen3-VL-2B-Instruct` base model.