picoxr/face-yolo-pipeline

Face YOLO Pipeline

This repository contains a prebuilt SpatialML pipeline package for running YOLO-based face detection with SecureMR on supported PICO devices.

The package is not only a model file. It bundles the serialized model, SecureML pipeline JSON files, package metadata, and rendering assets needed by an application to construct and run the full pipeline through the utility loader.

Package file

The main artifact is:

face-yolo-pipeline.zip

Extracting the zip produces a folder named face-yolo-pipeline/ with this structure:

face-yolo-pipeline/
├── manifest.json
├── gltf/
│   └── frame.gltf
├── model/
│   ├── model.json
│   ├── yolov8n-face.serialized.bin
│   └── yolov8n-face.serialized.json
└── pipeline/
    ├── face_detection_pipeline.json
    └── face_display_pipeline.json

What is included

• manifest.json: package metadata. It declares the pipeline entries, model paths, labels, detection tensor name, detection layout, thresholds, and runtime hints.
• model/yolov8n-face.serialized.bin: serialized SecureMR/QNN model binary.
• model/yolov8n-face.serialized.json: model tensor metadata for the serialized model.
• model/model.json: model loader metadata, including model name, input/output aliases, shapes, encoding types, engine type, and runtime configuration.
• pipeline/face_detection_pipeline.json: SecureMR pipeline definition for camera input, preprocessing, model inference, and JavaScript post-processing.
• pipeline/face_display_pipeline.json: SecureMR pipeline definition for consuming detection output, projecting image-space face coordinates into 3D, and rendering a face-frame overlay.
• gltf/frame.gltf: glTF asset rendered by the display pipeline.

How to use the package

Native SecureMR

Place the extracted face-yolo-pipeline/ folder in your app's assets directory. For example, in an Android project:

app/src/main/assets/face-yolo-pipeline/

For a native SecureMR integration, your C++ code should refer to the folder by its relative asset path:

constexpr const char* kModelPackageAssetPath = "face-yolo-pipeline";

Then load the package with the native SecureMR utility loader:

SecureMR::ModelPackagePipelineBundle bundle;
std::string error;

const bool loaded = SecureMR::SecureMrUtils::LoadModelPackagePipelinesFromAssets(
    kModelPackageAssetPath,
    frameworkSession,
    {},
    bundle,
    error);

In native code, LoadModelPackagePipelinesFromAssets(...) reads manifest.json, loads each referenced pipeline JSON file, patches model operators to use the packaged serialized model, creates shared global tensor bindings between pipelines, and binds packaged glTF assets through pipeline placeholders.

After loading, submit the package pipelines in sequence. This package contains:

1. detection: reads VST camera tensors, runs the Face YOLO model, and writes the post_det detection tensor.
2. display: consumes post_det, projects the detection into camera/world space, and renders the frame.gltf face-frame overlay.

Example SecureMR sample

An example application is available in the SecureMR samples repository:

• SecureMR Samples
• Face YOLO Pipeline sample

The native sample shows how to download or place this package in app assets and then load it with SecureMrUtils::LoadModelPackagePipelinesFromAssets(...).

Device support

This package is intended for applications running on supported PICO devices. It requires a runtime environment that supports the bundled pipeline JSON files and serialized model.

License

Apache License 2.0.