README.md

metadata

title: Sign Language Bridge
emoji: 🤟
colorFrom: indigo
colorTo: purple
sdk: static
app_file: index.html
short_description: ASL video to English translation (Qwen3-VL-2B LoRA)
license: apache-2.0
models:
  - mamounyosef/sign-language-bridge
  - Qwen/Qwen3-VL-2B-Instruct
datasets:
  - how2sign
  - openasl
tags:
  - sign-language
  - sign-language-translation
  - asl
  - video-text-to-text

sign-language-bridge — ASL → English

⏳ Awaiting GPU hardware

The Gradio app in this repo (app.py, preprocessing.py) is complete and written for ZeroGPU. It is not running yet: hosting a Gradio Space requires either PRO or a free-account ZeroGPU slot, and this account is currently under the 30-day age threshold for the free allowance.

To switch it on, replace the frontmatter above with:

sdk: gradio
sdk_version: 6.24.0
app_file: app.py
python_version: "3.12"
startup_duration_timeout: 45m

and attach zero-a10g hardware — either by subscribing to PRO, waiting out the 30 days, or being awarded a community GPU grant. No code changes are needed.

Demo for mamounyosef/sign-language-bridge, a multi-tier LoRA / RSLoRA fine-tune of Qwen/Qwen3-VL-2B-Instruct for continuous American Sign Language → English translation, trained on How2Sign + OpenASL.

Upload or record a 1–15 second ASL clip and the Space returns an English translation, plus the preprocessed clip the model actually saw.

Why this Space reproduces the training preprocessing

The adapter was trained with three preprocessing stages applied to every split, so inference has to reproduce them or the model receives an out-of-distribution input and quality collapses. preprocessing.py ports them from the project repo:

Stage	What it does	Ported from
Pose-guided signer crop	MediaPipe PoseLandmarker; union of upper-body + hand landmarks across the clip, padded 25 %, snapped to a multiple of 32	data_code/signer_cropper.py
CLAHE	Contrast equalisation on the L channel in LAB (clip 2.0, 8×8 tiles)	Qwen3VLCollator._apply_clahe_opencv
Landmark overlay	RTMPose Wholebody; 6 upper-body joints + 21 keypoints per hand drawn as a 1 px skeleton	data_code/21_extract_landmarks.py, Qwen3VLCollator._apply_landmark_overlay

Each stage has a UI toggle so you can see how much the model leans on it.

Video sampling and generation settings match the evaluated checkpoint (step 4,610) exactly:

• 20 fps, min_pixels 4,096, max_pixels 184,320, total_pixels 20,971,520
• System prompt: "You are a sign language translator."
• User prompt: "Translate this American Sign Language video into English."
• Beam size 5, length penalty 0.6, no-repeat n-gram 4, repetition penalty 1.1, max 32 new tokens

Limitations

This is a research preview. On the author's How2Sign test partition (944 clips, a custom 90/5/5 split — not the official How2Sign split, so the numbers are not comparable to published results):

BLEU-1	BLEU-4	chrF	ROUGE-L	METEOR	WER
19.76	1.64	17.42	10.43	9.71	112.5 %

Output is fluent English in the register of the target captions and often topically appropriate, but frequently disagrees with the reference word for word — and can be confidently wrong. Do not use it where a mistranslation could cause harm (medical, legal, safety-critical, or emergency settings). The model has seen almost only How2Sign / OpenASL signers, framings, and lighting, and generalises poorly outside them.

License

Apache 2.0, matching the adapter and the Qwen3-VL-2B-Instruct base model.