---
license: cc-by-nc-4.0
library_name: pytorch
pipeline_tag: depth-estimation
tags:
  - depth-estimation
  - monocular-depth
  - face
  - coreml
  - apple-silicon
  - knowledge-distillation
  - depth-anything-v2
base_model: depth-anything/Depth-Anything-V2-Large
datasets:
  - CelebAMask-HQ
language:
  - en
---

# FaceDepth

Face-specialized monocular depth estimation. One photo in, sharp facial relief out, with no depth sensor.

General depth models train on scenes and interiors, so they flatten a face into a smooth blob. FaceDepth fine-tunes Depth Anything V2-Large on 30,000 CelebA-HQ faces, distilling the boundary-accurate Depth Pro teacher under three losses guided by CelebAMask-HQ face parsing. It resolves the eyelid crease, the nostril rim, the lip contour, and the hairline.

Paper and code: [github.com/AristidesAI/FaceDepth](https://github.com/AristidesAI/FaceDepth)

## Results

Measured on 500 held-out CelebA-HQ faces.

| metric | pretrained DA2-Large | FaceDepth | change |
|---|---|---|---|
| face-region SSI-MAE | 0.02764 | **0.00906** | **-67%** |
| depth-edge F1 vs teacher | 0.717 | **0.882** | **+23%** |
| edge recall vs teacher | 0.745 | **0.873** | +17% |
| edge precision vs teacher | 0.694 | **0.893** | +29% |

Edge metrics are density-matched at 5% of in-face gradient pixels with a 2-pixel tolerance, so a blurry model cannot win by spreading weak gradients across the face. Both recall and precision rise, so the student finds the teacher's edges and stops inventing edges the teacher does not have.

## Files

| file | format | size | notes |
|---|---|---|---|
| `FaceDepth_step15792.pt` | PyTorch | 2.5 GB | original best checkpoint, contains `model`, `ema_model`, `conf_head`. Use `ema_model`. |
| `coreml/FaceDepth_fp32.mlpackage` | Core ML | 1.2 GB | unquantized reference, 5.4 fps |
| `coreml/FaceDepth_fp16.mlpackage` | Core ML | 668 MB | **realtime, 40.5 fps, runs on the Neural Engine** |
| `coreml/FaceDepth_int8.mlpackage` | Core ML | 335 MB | 39.4 fps, smallest, runs on the GPU |

All three Core ML exports correlate at 1.00000 against the PyTorch reference. Benchmarks are 392x518 input on an Apple-silicon laptop with `ComputeUnit.ALL`, mean over 20 runs after warmup.

On this hardware int8 buys a 2x size reduction rather than speed. Its advantage is the app bundle. The ranking may differ on iPhone, where the Neural Engine is relatively stronger, and we have not measured that.

## Usage

### Core ML (recommended, realtime)

```python
import numpy as np, coremltools as ct
from PIL import Image

model = ct.models.MLModel("coreml/FaceDepth_fp16.mlpackage",
                          compute_units=ct.ComputeUnit.ALL)
img = Image.open("face.jpg").convert("RGB").resize((392, 518))
disp = np.asarray(model.predict({"image": img})["depth"]).reshape(518, 392)
# disp is inverse depth: larger = nearer
```

Input is a 392x518 portrait RGB image. ImageNet normalization is folded into the graph, so pass raw pixels. Output is relative inverse depth, where larger means nearer.

### PyTorch

```python
import torch, sys
sys.path.insert(0, "third_party/DepthAnythingV2")
from depth_anything_v2.dpt import DepthAnythingV2

m = DepthAnythingV2(encoder="vitl", features=256,
                    out_channels=[256, 512, 1024, 1024])
ck = torch.load("FaceDepth_step15792.pt", map_location="cpu", weights_only=True)
m.load_state_dict(ck["ema_model"])
m = m.to("mps").eval()
```

The checkpoint also holds a `conf_head`, a training-time per-pixel confidence head. Inference does not need it and the Core ML exports drop it.

### Normalizing the output for display

The model returns relative inverse depth with an arbitrary scale. Normalize within the face rather than the whole frame. A whole-frame min-max collapses the face's range as soon as a distant background enters the shot, which reads as a black or washed-out face.

```python
lo, hi = np.percentile(disp, 2), np.percentile(disp, 98)
norm = np.clip((disp - lo) / (hi - lo), 0, 1)   # 1 = nearest
```

## How it was trained

**Teacher.** Depth Pro at native 1024 px labels 30,000 CelebA-HQ faces, stored as inverse depth. The teacher sets the ceiling on the student's sharpness, and this choice is the dominant lever in the whole recipe.

**Masks.** The 19 CelebAMask-HQ classes collapse into a head foreground mask, a per-pixel feature weight (eyes and brows highest, nose and lips medium, skin and hair base), and a boundary map of feature edges.

**Losses.**

- Foreground-restricted scale-and-shift-invariant trimmed MAE. Aligning scale over the head alone spends the model's dynamic range on facial relief instead of the background.
- Feature-weighted multi-scale gradient matching on the depth residual. This is the term that produces sharpness.
- Confidence-weighted regression with a learned per-pixel weight, so the student discounts pixels the teacher labels unreliably.
- A boundary term at parsed feature edges, pushing crisp depth steps to the lid line, lip, nostril, and hairline.

**Training.** Multi-resolution crops at 518, 700, and 910 px. AdamW, head and encoder learning rates 2e-5 and 2e-6, cosine schedule, gradient clipping at 1.0, fp32, gradient checkpointing, EMA 0.999. 17,000 steps on one Apple-silicon laptop, stopped on a validation plateau.

## Limitations

This model optimizes single-image sharpness. The temporal-consistency loss came out of the recipe to get there, so live video flickers more than a temporally trained model would. For offline video, smooth the normalization range across frames, and optionally the depth itself, which trades flicker for ghosting under fast motion.

Distillation caps detail at the teacher. The claim here is sharp feature relief and boundaries, not sub-millimeter texture, and per-eyelash depth is beyond what any current monocular teacher resolves.

Training data is CelebA-HQ, which is centered, well-lit, and limited in pose, occlusion, and demographic diversity relative to in-the-wild use. Performance by demographic group has not been measured. Evaluate before deploying on populations or capture conditions that differ from CelebA-HQ.

Loss-term and resolution ablations have not been run, so the contribution attributed to each individual loss term rests on the design argument rather than measured deltas.

## License and provenance

Released under **CC-BY-NC-4.0**, non-commercial research use.

This is a derivative of [Depth Anything V2-Large](https://huggingface.co/depth-anything/Depth-Anything-V2-Large), which is CC-BY-NC-4.0. It trains on pseudo-labels from [Apple Depth Pro](https://github.com/apple/ml-depth-pro) and on [CelebAMask-HQ](https://github.com/switchablenorms/CelebAMask-HQ), whose terms restrict use to non-commercial research and education. Honor the upstream terms of all three.

## Citation

```bibtex
@software{facedepth2026,
  title  = {FaceDepth: Face-Specialized Monocular Depth by Distilling a
            Boundary-Accurate Teacher under Segmentation-Guided Losses},
  author = {Lintzeris, Aristides},
  year   = {2026},
  url    = {https://huggingface.co/a-ml/FaceDepth}
}
```

Please also cite the upstream work this builds on: Depth Anything V2 (arXiv:2406.09414), Depth Pro (arXiv:2410.02073), MiDaS (arXiv:1907.01341), DINOv2 (arXiv:2304.07193), DPT (arXiv:2103.13413), and CelebAMask-HQ (arXiv:1907.11922).