dkx2077/qwen3-vl-4b-multimodal-jlens-vqav2-100

Qwen3-VL-4B Multimodal J-lens on VQAv2

This repository hosts a fitted multimodal Jacobian lens for Qwen/Qwen3-VL-4B-Instruct, produced by the J-space-in-VLM project.

The lens transports an intermediate residual vector h_l into the final residual basis and decodes it with the model's own norm + lm_head:

J-lens_l(h_l) = lm_head(norm(J_l @ h_l))
J_l = E[d h_final / d h_l]

Files

• lens.pt: averaged JacobianLens file for readout.
• lens.ckpt.pt: resumable fitting checkpoint with raw jacobian_sum.
• metadata.json: fit configuration and layer metadata.
• config.json: lightweight lens metadata.

This repository is not a standalone Transformers model. config.json describes the J-lens artifact; load the weights with the jlens code path below.

Fit Details

• Base model: Qwen/Qwen3-VL-4B-Instruct
• Dataset: local 1000-example VQAv2 val2014 subset
• Fit split: fit_900.jsonl
• Requested samples: 100
• Completed samples in this uploaded checkpoint: 49
• Skipped samples: 393282000 was skipped after CUDA OOM
• Checkpoint next_idx: 50
• Source layers: every 2 layers, [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34]
• Target layer: 35
• Position scope: all_nonfinal

Important: this upload is named for the 100-sample fitting run, but fitting was interrupted before all 100 requested samples completed. lens.pt is averaged over 49 completed samples only.

Loading

import jlens

lens = jlens.JacobianLens.from_pretrained(
    "dkx2077/qwen3-vl-4b-multimodal-jlens-vqav2-100",
    filename="lens.pt",
)

Use the code in dkx2077/J-space-in-VLM for the Qwen3-VL multimodal adapter and evaluation scripts.