ppark-ai/tachand-v-foundation-v2-4a

TacHand V Foundation v2.4-A — Model Card

Model ID (planned): ppark-ai/tachand-v-foundation-v2-4a License: Apache 2.0 (weights) + CC-BY 4.0 / research-only (eval data references; see corpus table) Pretrain date: 2026-05-16 Maintainer: ppark-ai

---

TL;DR

Frozen-V + adapter contact-aware vision foundation. v2.4-A is the v2 release winner: frozen DINOv2-L (initialised from v2.3) + trainable T branches / v_proj / sensor token + NEW corpora (TAG, Sparsh, TacQuad) at small cap, lifts to 82.3%.

Base: DINOv2-L (Meta, 300M params), frozen Initialisation: v2.3 release-candidate checkpoint Trainable: v_proj (1024→512), T branches, sensor-type embedding Pretrained on: 153,562 paired V+T frames across 9 corpora (v1 mix replay + TAG/Sparsh/TacQuad capped) Open-loop FSL accuracy: 82.3% k=10 nn=1 (5 seeds, 10-class VTDexManip n=200, matched v1 audit protocol), +4.4pp vs v1, +2.8pp vs v2.3. Latent dim: 512 (projected) / 1024 (raw DINOv2 output) Input: 112×112 RGB

This is Layer 2 of the Open Tactile Stack. v2.4-A supersedes v2.3 as the recommended Layer 2 checkpoint for downstream ContactAdapter / VLA wrappers.

Why frozen-V + adapter. v2.1 (full NEW-corpora joint mix) and v2.2 (capped NEW-corpora joint mix) regressed VTDex by 27–38 pp under joint V-trunk training. Codex agent diagnosis: heterogeneous NEW corpora dilute the V representation through shared-trunk capacity contention. v2.4-A freezes the V trunk (preserving the v2.3 representation) and routes NEW-corpus signal through the trainable T branches, v_proj, and sensor token only. Empirically confirmed: 82.3 % vs v2.1 catastrophic 50 %. See docs/validation/v2_new_corpora_poison_failure_audit.md.

---

Architecture

RGB ──► DINOv2-L (FROZEN, v2.3-init) ──► v_proj (1024→512, TRAIN) ──► z_v
                                                                       │
Tactile (per-sensor) ──► T branch (TRAIN) ──► z_t ─────────────────────┤
Sensor-type ──────────► sensor token (TRAIN) ──► z_s ──────────────────┤
                                                                       │
                                                  InfoNCE(z_v, z_t, z_s)

The V trunk is bit-for-bit identical to v2.3; only the projection, modality branches, and sensor token receive gradient. This is the same decoupling pattern as Layer 3 ContactAdapter (a_final = a_base + gate * clip(delta)) — TacHand's signature design carried into Layer 2 pretrain.

---

Usage (3 lines)

from tachand_tools.foundation import VFoundation

v = VFoundation.load("ppark-ai/tachand-v-foundation-v2-4a")
embed = v.encode(rgb_image)                  # → torch.Tensor (512,)
embeds = v.encode_batch([img1, img2, img3])  # → (3, 512)

Frozen by default. No grad. Use as feature extractor for downstream tasks.

---

Pretrain details

Field	Value
Backbone	dinov2_vitl14 (Meta, 300M params), frozen
Initialisation	v2.3 RC checkpoint (results/v_foundation_v2.3/checkpoint.pt)
Trainable	v_proj (Linear 1024→512), T branches, sensor token
Projection	Linear(1024 → 512)
Image size	112 (DINOv2 14×8 patches)
Optimizer	AdamW
Batch size	64 per corpus, round-robin
Learning rate	1e-4
Weight decay	1e-5
Epochs	4
Loss	InfoNCE V↔T_image / V↔T / V↔L contrastive (β-arch N-modality)
Modality dropout	0.2
Wall time	~1 h on RTX 5090

Training data (153,562 paired V+T frames across 9 corpora)

Corpus	Frames	Modality	Role	License
VTDexManip (Zhang 2025)	6,700	RGB + 3-axis force	v1 mix replay	research-only
TacHand sim	25,000	RGB (synthetic) + force-vec	v1 mix replay	internal
FreeTacMan features	33,000	features only	v1 mix replay	internal
TVL SSVTP (Fu 2024 ICML)	4,000	RGB + DIGIT image	v1 mix replay	CC-BY 4.0
TVL HCT (Fu 2024)	5,000	RGB + DIGIT image (imdesc captions)	v1 mix replay	CC-BY 4.0
ObjectFolder Real (Gao 2022)	24,000	RGB + GelSight	v1 mix replay	CC-BY 4.0
Touch & Go (Yang 2022)	833	RGB + GelSight	NEW (capped)	CC-BY 4.0
Sparsh touch_slide (Suresh 2024)	64,000	DIGIT image	NEW (capped)	CC-BY 4.0
TacQuad (Wu 2025)	1,829	multi-sensor	NEW (capped)	research-only

Total: 153,562 frames. NEW corpora are capped so they cannot dominate the round-robin batch schedule — combined with the frozen V trunk, this is the architectural fix for the v2.1 / v2.2 poison.

Training command

python scripts/v2/pretrain_v_foundation_v2.py \
    --vtdex-episodes 1500 \
    --sim-episodes 1000 \
    --ftm-root data/freetacman_features_d256_v2 \
    --tvl-ssvtp-root data/tvl_full/tvl_dataset/ssvtp \
    --tvl-hct-root data/tvl_full/tvl_dataset/hct \
    --ofreal-root data/objectfolder_real/extracted --ofreal-max-obj 50 \
    --tag-root data/touch_and_go --tag-max-frames 833 \
    --sparsh-root data/sparsh/touch_slide --sparsh-max-pkls 64 \
    --tacquad-root data/tacquad --tacquad-max-frames 1829 \
    --epochs 4 --batch-size 64 --lr 1e-4 \
    --embed-dim 512 --vision dinov2-l --image-size 112 \
    --sensor-token \
    --freeze-vision \
    --init-ckpt results/v_foundation_v2.3/checkpoint.pt \
    --output results/v_foundation_v2.4a_frozen

Flags landed in commit cdb24f4 (--freeze-vision, --freeze-v-proj, --init-ckpt).

Reproducibility

Seed	k=10 nn=1 (n=200)
0	0.830
1	0.780
2	0.800
3	0.825
4	0.880
mean ± std	0.823 ± 0.038

Per-class audit (3 seeds, 10-class mean): 80.8 ± 5.3 %.

Checkpoint: /home/patrick/results/v_foundation_v2.4a_frozen/checkpoint.pt (1.83 GB) Checkpoint SHA: <v2.4-A ckpt commit pending>

---

Evaluation

Protocol: 10-class VTDexManip open-loop FSL, k-NN retrieval, 5 seeds × n=200, matched v1 audit protocol.

Comparison	Δ (pp)	Verdict
v2.4-A − v1 (matched 5-seed n=200)	+4.4	win
v2.4-A − v2.3 (matched 5-seed n=200)	+2.8	win
v2.4-A − v2.1 (NEW corpora joint, 4 ep)	+31.9	NEW-corpus poison removed
v2.4-A − v2.2 (NEW corpora capped joint, 1 ep)	+42.1	NEW-corpus poison removed

Per-class table (3 seeds, n=200, mean ± std)

Class	v2.4-A	v1	Δ vs v1 (pp)
Pressing	68.3 %	56.7 %	+11.6 ★
Scissors	60.0 %	56.7 %	+3.3
(8 remaining classes)	—	—	covered by overall mean
Overall (10-class mean)	80.8 ± 5.3 %	76.3 %	+4.5

Worst-class lift (Scissors / Pressing) is the headline robustness signal: v1's two structural weaknesses both move up.

Cross-corpus held-out (held_out_30pct vs raw DINOv2-L)

Corpus	v2.4-A Δ vs raw DINOv2-L (pp)	Status
TAG	−3.7	partial — still negative
Sparsh touch_slide	+0.0	saturated
TacQuad	−1.5	partial — still negative
NeuralFeels	+0.0	trivial (truly-unseen RGB)

Cross-corpus is not yet positive for TAG / TacQuad. v2.4-A is an in-domain win with cross-corpus partial recovery vs v2.1 / v2.2 (which were larger negatives); the +5 pp cross-corpus gate remains open for v2.5.

---

Intended use

1. Frozen feature extractor for V/T downstream tasks (FSL, classification, retrieval).
2. Layer 2 of Open Tactile Stack: input to ContactAdapter + robot company VLA.
3. Recommended Layer 2 checkpoint: v2.4-A supersedes v1 / v2.3 for new downstream work.

Out-of-scope

• Cross-corpus claims on TAG / TacQuad — Δ is still negative vs raw DINOv2-L on those held-outs; use raw DINOv2-L if your evaluation is TAG- or TacQuad-only.
• Direct fine-tuning of the V trunk (it's frozen by design; the architectural reason is the v2.1 / v2.2 poison).
• Closed-loop policy on its own (use ContactAwareWrapper + ContactAdapter).
• Sensor-specific calibration (use SensorAdapter for sensor → standard latent).

Limitations

• Cross-corpus partial recovery. TAG −3.7 pp, TacQuad −1.5 pp vs raw DINOv2-L. v2.5 next steps: per-corpus adapter heads, larger NEW-corpus cap with stronger frozen-V isolation, or sensor-token re-balancing.
• NeuralFeels +0 pp is trivial: truly-unseen RGB held-out, both arms at chance under the current protocol; not a v2.4-A signal either way.
• Open-loop FSL ≠ closed-loop success (memory: MSE↛success). Use governance gating.
• Headline is on a 10-class VTDexManip subset; broader-class evaluation pending.
• n = 5 seeds is the floor for v2.4-A; per-class table is n = 3.
• Wall time, training data, and ckpt size were measured on the dev GPU (RTX 5090); HF Hub publish + checkpoint SHA fill follow once private staging clears.

---

v2 ablation history

Run	Mix	Epochs	VTDex k10 nn=1	Verdict
v1	v1 corpora (7)	4	76.3–77.9 %	canonical baseline
v2.0	RAM OOM during data load	—	—	abandoned
v2.1	v1 + NEW corpora (full mix)	4	50.4 %	catastrophic (−26 to −27 pp)
v2.2	v1 + NEW corpora (capped)	1	40.2 %	further regression
v2.3	v1-mix-only	4	79.5 %	RC (v1 +1.6 pp)
v2.4-A ★	frozen-V (v2.3-init) + NEW corpora capped	4	82.3 %	release winner

Full audit: docs/validation/v2_new_corpora_poison_failure_audit.md. Process post-mortem: docs/release/v_foundation_v2_4a_failure_to_success_narrative.md.

---

Citation

@misc{park2026tachand_v2_4a,
  author       = {Park, Soohwan and Patrick},
  title        = {{TacHand V Foundation v2.4-A}: Frozen DINOv2-L + Adapter
                  for Heterogeneous Tactile Corpora},
  year         = {2026},
  month        = may,
  howpublished = {\url{https://huggingface.co/ppark-ai/tachand-v-foundation-v2-4a}},
  note         = {Open Tactile Stack, Layer 2}
}

Built on top of v2.3 (which is built on v1); see docs/release/v_foundation_v1_card.md and docs/release/v_foundation_v2_3_model_card.md for full ancestry and underlying DINOv2 / VTDexManip / TVL / ObjectFolder / Touch-and-Go / Sparsh / TacQuad citations.

---

Layer 3 ContactAdapter (wrapper evidence)

Frozen v2.4-A V Foundation + ContactResidualHead (LayerNorm + 2-layer MLP, ~165 K trainable params) + ActionSpec L2 gate. Strong-eval: 8 DAgger iters × 30 rollouts/iter × 3 tasks × single seed = 720 paired wrapped-vs-baseline rollouts.

Task	Wrapped peak / mean (iter 4-8)	Baseline peak / mean (iter 4-8)	Lift (mean)
pick_and_place	100 % / 98.0 % (4 iters at 100)	93 % / 88.6 %	+9.4 pp robust
carry_full_cup	10 % / 6.0 % (9/150)	0 % / 0.0 % (0/240 all 8 iters)	+6.0 pp consistent
wipe_table	0 % / 0 %	0 % / 0 %	0 pp (sim physics limit)

Architecture: frozen v2.4-A trunk → z_v (512) + proprio (16) + phase (1) → LayerNorm → MLP (192 hidden, SiLU, 2 layers) → action chunk K=8 × action_dim=16. Trainable parameters ~165 K.

ActionSpec gate: L2_EE_GRIPPER_RESIDUAL with max_residual_norm = 0.20 enforced. can_inject_residual(action_spec) is the hard gate — unknown action semantics fall back to a_base. Per inference: residual = head_out[0] - a_base; a_final = a_base + clip(residual, ±0.05).

Caveats (strong-eval-updated):

1. Statistical sample size. Single seed, 720 paired rollouts (30 × 8 × 3); consistent over 5 post-warmup iters for carry_full_cup (every iter > 0) and pick_and_place (4 iters at 100 %), not single-iter peak. Multi-seed replication is the next promotion gate.
2. wipe_table 0 % / 0 % is the documented sim physics limit (oracle-only task); baseline also fails — wrapper does not regress.
3. pick_and_place baseline 88.6 % mean leaves ≤11 pp absolute headroom; the +9.4 pp lift is robust over that bounded range.
4. a_base = cur_proprio is a sim placeholder. Deployment uses a real VLA's action as a_base; the residual composition is the same shape but transfer is unverified.
5. Init pretrain uses zero-RGB embedding (sim H5 demos predate V Foundation, do not store RGB); 8 DAgger iters with real per-step RGB mitigate this — the carry_full_cup lift appears consistently across iters 4-8, well after the first relabel pass.

Full method, training command, and per-iter table: docs/release/v_foundation_v2_4a_contact_adapter_evidence.md. Strong-eval source: ~/results/contact_adapter_v2_4a_strong/train_summary.json and ~/contact_adapter_v2_4a_strong.log (GPU server).