Hugging Face's logo

baulab
/
Qwen3-VL-2B-Instruct-GazeHeads-AllLayers-ONNX

Image-Text-to-Text
Transformers.js
ONNX
qwen3_vl
webgpu
interpretability
attention-steering
conversational
Model card Files
xet
 Community
Qwen3-VL-2B-Instruct-GazeHeads-AllLayers-ONNX / README.md
RohitGandikota's picture
RohitGandikota
all-layers decomposed: all 448 heads steerable
587c336 verified
preview code
|
Raw
History Blame Contribute Delete
1.1 kB
metadata 
license: apache-2.0
base_model: Qwen/Qwen3-VL-2B-Instruct
library_name: transformers.js
pipeline_tag: image-text-to-text
tags:
  - onnx
  - webgpu
  - interpretability
  - attention-steering

Qwen3-VL-2B-Instruct — Gaze-Head Steering, ALL LAYERS (ONNX, WebGPU)

Same as baulab/Qwen3-VL-2B-Instruct-GazeHeads-ONNX but with every transformer layer decomposed, so the gaze_head_mask / gaze_sign steering inputs are honored for all 448 attention heads (28 layers x 16 heads), not just the gaze-hosting layers. Used for the interactive demo control conditions (steer all heads / random heads).

Two extra decoder inputs implement the paper intervention (zeros = original):

  • gaze_head_mask (28, 16) float16 — 1.0 selects a head to steer
  • gaze_sign (total_seq_len,) float16 — additive pre-softmax attention bias

Inference is slower than the 7-layer build (all layers use decomposed attention instead of fused GroupQueryAttention). Weight data is bit-identical to the upstream export; only the decoder graph differs.