LivePortrait
talking-head
avatar
real-time
conversational-ai
4d-avatar
eden-protocol
lip-sync
tts
asr
Instructions to use AIBRUH/eden-os with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LivePortrait
How to use AIBRUH/eden-os with LivePortrait:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
EDEN OS v1.0.1 — Skin Realism Agent + VRAM Strategy + README
Browse files- README.md +109 -0
- eden_os/genesis/__init__.py +3 -1
- eden_os/genesis/skin_realism_agent.py +623 -0
- scripts/save_versioned.sh +132 -0
README.md
ADDED
|
@@ -0,0 +1,109 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: EDEN OS
|
| 3 |
+
emoji: 🌿
|
| 4 |
+
colorFrom: black
|
| 5 |
+
colorTo: gold
|
| 6 |
+
license: apache-2.0
|
| 7 |
+
tags:
|
| 8 |
+
- talking-head
|
| 9 |
+
- avatar
|
| 10 |
+
- real-time
|
| 11 |
+
- conversational-ai
|
| 12 |
+
- 4d-avatar
|
| 13 |
+
- eden-protocol
|
| 14 |
+
- liveportrait
|
| 15 |
+
- lip-sync
|
| 16 |
+
- tts
|
| 17 |
+
- asr
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
# EDEN OS — 4D Conversational Avatar Operating System
|
| 21 |
+
|
| 22 |
+
**Version 1.0 | Phase One | Codename: OWN THE SCIENCE**
|
| 23 |
+
|
| 24 |
+
Upload any 2D portrait → Get a photorealistic talking avatar that converses in real-time with perfect lip sync, emotional expression, and knowledge of your content.
|
| 25 |
+
|
| 26 |
+
## What is EDEN OS?
|
| 27 |
+
|
| 28 |
+
EDEN OS is a **headless operating system** — not an app. It's a backend engine that exposes a universal API. Any frontend (React, Gradio, mobile, VR) plugs into it.
|
| 29 |
+
|
| 30 |
+
## Architecture: 7 Specialized Engines
|
| 31 |
+
|
| 32 |
+
| Engine | Agent | Role |
|
| 33 |
+
|--------|-------|------|
|
| 34 |
+
| **Genesis** | Agent 1 | Portrait processing, Eden Protocol validation, skin realism |
|
| 35 |
+
| **Voice** | Agent 2 | ASR (Whisper), TTS (Kokoro/CosyVoice2), voice cloning, emotion routing |
|
| 36 |
+
| **Animator** | Agent 3 | LivePortrait real-time animation, idle generation, state machine |
|
| 37 |
+
| **Brain** | Agent 4 | Claude Sonnet streaming, persona management, memory, RAG context |
|
| 38 |
+
| **Conductor** | Agent 5 | Pipeline orchestration, latency enforcement, error recovery |
|
| 39 |
+
| **Gateway** | Agent 6 | FastAPI REST API, WebSocket streaming, video encoding |
|
| 40 |
+
| **Scholar** | Agent 7 | YouTube/audiobook/URL ingestion, ChromaDB RAG, knowledge graph |
|
| 41 |
+
|
| 42 |
+
## The Eden Protocol
|
| 43 |
+
|
| 44 |
+
Every generated frame must pass the **0.3 deviation rule** — skin texture fidelity measured in LAB color space with Gabor filter banks. This eliminates the "plastic skin" artifact, especially on melanin-rich skin tones.
|
| 45 |
+
|
| 46 |
+
**Skin Realism Agent**: Built-in skill agent for:
|
| 47 |
+
- Pore-level micro-texture synthesis
|
| 48 |
+
- Subsurface scattering simulation
|
| 49 |
+
- Melanin-aware color correction
|
| 50 |
+
- Emotion-driven skin response (blush, pallor)
|
| 51 |
+
- Identity marker preservation (freckles, moles, beauty marks)
|
| 52 |
+
|
| 53 |
+
## API
|
| 54 |
+
|
| 55 |
+
```bash
|
| 56 |
+
# Health check
|
| 57 |
+
curl https://AIBRUH-eden-os.hf.space/api/v1/health
|
| 58 |
+
|
| 59 |
+
# Create session
|
| 60 |
+
curl -X POST /api/v1/sessions -d '{"template": "medical_office"}'
|
| 61 |
+
|
| 62 |
+
# Inject knowledge
|
| 63 |
+
curl -X POST /api/v1/knowledge/ingest -d '{"type": "youtube", "url": "..."}'
|
| 64 |
+
|
| 65 |
+
# WebSocket bi-directional stream
|
| 66 |
+
ws://host/api/v1/sessions/{id}/stream
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
## 3-Tier VRAM Strategy
|
| 70 |
+
|
| 71 |
+
| Tier | Storage | Purpose |
|
| 72 |
+
|------|---------|---------|
|
| 73 |
+
| **HOT** | GPU VRAM | Active models (LivePortrait 4GB + TTS 0.5GB + VAD 0.1GB) |
|
| 74 |
+
| **WARM** | Seagate 5TB / Local SSD | Pre-downloaded weights, instant swap |
|
| 75 |
+
| **COLD** | HuggingFace Hub (1TB) | Persistent cloud cache |
|
| 76 |
+
|
| 77 |
+
## Quick Start
|
| 78 |
+
|
| 79 |
+
```bash
|
| 80 |
+
git clone https://github.com/tyronne-os/EDEN-OS
|
| 81 |
+
cd EDEN-OS
|
| 82 |
+
python -m venv .venv && source .venv/bin/activate
|
| 83 |
+
pip install -r requirements.txt
|
| 84 |
+
python app.py
|
| 85 |
+
# Open http://localhost:7860
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
## Hardware Profiles
|
| 89 |
+
|
| 90 |
+
| Tier | GPU | Animation | TTS | LLM |
|
| 91 |
+
|------|-----|-----------|-----|-----|
|
| 92 |
+
| H100 (80GB) | HunyuanAvatar + LivePortrait | CosyVoice2 | Claude Sonnet |
|
| 93 |
+
| RTX 4090 (24GB) | LivePortrait | CosyVoice2 | Claude Sonnet |
|
| 94 |
+
| RTX 3090 (24GB) | LivePortrait | Kokoro | Qwen3 8B |
|
| 95 |
+
| CPU Only | LivePortrait (reduced fps) | Kokoro | BitNet 3B |
|
| 96 |
+
|
| 97 |
+
## Built With
|
| 98 |
+
|
| 99 |
+
- **LivePortrait** (KwaiVGI) — 78fps implicit keypoint animation
|
| 100 |
+
- **Whisper** (OpenAI) — real-time ASR
|
| 101 |
+
- **Claude Sonnet** (Anthropic) — conversational reasoning
|
| 102 |
+
- **ChromaDB** + **sentence-transformers** — RAG knowledge retrieval
|
| 103 |
+
- **FastAPI** + **aiortc** — API + WebRTC streaming
|
| 104 |
+
|
| 105 |
+
---
|
| 106 |
+
|
| 107 |
+
**OWN THE SCIENCE.**
|
| 108 |
+
|
| 109 |
+
*Built by TJ LSU DAD + Claude Code Agents*
|
eden_os/genesis/__init__.py
CHANGED
|
@@ -21,6 +21,7 @@ from eden_os.genesis.portrait_engine import PortraitEngine
|
|
| 21 |
from eden_os.genesis.eden_protocol_validator import EdenProtocolValidator
|
| 22 |
from eden_os.genesis.latent_encoder import LatentEncoder
|
| 23 |
from eden_os.genesis.preload_cache import PreloadCache
|
|
|
|
| 24 |
|
| 25 |
|
| 26 |
class GenesisEngine(IGenesisEngine):
|
|
@@ -38,6 +39,7 @@ class GenesisEngine(IGenesisEngine):
|
|
| 38 |
self._validator = EdenProtocolValidator()
|
| 39 |
self._encoder = LatentEncoder(latent_dim=latent_dim)
|
| 40 |
self._cache = PreloadCache(num_seeds=num_idle_seeds)
|
|
|
|
| 41 |
logger.info("GenesisEngine initialised (latent_dim={}, idle_seeds={})",
|
| 42 |
latent_dim, num_idle_seeds)
|
| 43 |
|
|
@@ -113,4 +115,4 @@ class GenesisEngine(IGenesisEngine):
|
|
| 113 |
logger.info("GenesisEngine closed")
|
| 114 |
|
| 115 |
|
| 116 |
-
__all__ = ["GenesisEngine"]
|
|
|
|
| 21 |
from eden_os.genesis.eden_protocol_validator import EdenProtocolValidator
|
| 22 |
from eden_os.genesis.latent_encoder import LatentEncoder
|
| 23 |
from eden_os.genesis.preload_cache import PreloadCache
|
| 24 |
+
from eden_os.genesis.skin_realism_agent import SkinRealismAgent
|
| 25 |
|
| 26 |
|
| 27 |
class GenesisEngine(IGenesisEngine):
|
|
|
|
| 39 |
self._validator = EdenProtocolValidator()
|
| 40 |
self._encoder = LatentEncoder(latent_dim=latent_dim)
|
| 41 |
self._cache = PreloadCache(num_seeds=num_idle_seeds)
|
| 42 |
+
self.skin_agent = SkinRealismAgent()
|
| 43 |
logger.info("GenesisEngine initialised (latent_dim={}, idle_seeds={})",
|
| 44 |
latent_dim, num_idle_seeds)
|
| 45 |
|
|
|
|
| 115 |
logger.info("GenesisEngine closed")
|
| 116 |
|
| 117 |
|
| 118 |
+
__all__ = ["GenesisEngine", "SkinRealismAgent"]
|
eden_os/genesis/skin_realism_agent.py
ADDED
|
@@ -0,0 +1,623 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
EDEN OS — Skin Realism Agent
|
| 3 |
+
Specialized skill agent for natural skin texture enhancement.
|
| 4 |
+
Ensures photorealistic skin that is indistinguishable from real human video.
|
| 5 |
+
|
| 6 |
+
This agent handles:
|
| 7 |
+
1. Pore-level micro-texture synthesis
|
| 8 |
+
2. Subsurface scattering simulation (SSS) for skin translucency
|
| 9 |
+
3. Melanin-aware color correction (works across ALL skin tones)
|
| 10 |
+
4. Specular highlight preservation (natural skin sheen, not plastic)
|
| 11 |
+
5. Fine hair and peach fuzz rendering
|
| 12 |
+
6. Skin imperfection preservation (freckles, moles, beauty marks, pores)
|
| 13 |
+
7. Dynamic skin response (blush, pallor, blood flow under emotion)
|
| 14 |
+
|
| 15 |
+
The agent works as a post-processing pass on every frame from the Animator,
|
| 16 |
+
ensuring the Eden Protocol's 0.3 deviation threshold is met with emphasis
|
| 17 |
+
on skin naturalness rather than just pixel accuracy.
|
| 18 |
+
"""
|
| 19 |
+
|
| 20 |
+
import asyncio
|
| 21 |
+
from dataclasses import dataclass, field
|
| 22 |
+
from typing import Optional
|
| 23 |
+
|
| 24 |
+
import cv2
|
| 25 |
+
import numpy as np
|
| 26 |
+
from loguru import logger
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
@dataclass
|
| 30 |
+
class SkinProfile:
|
| 31 |
+
"""Extracted skin characteristics from the reference portrait."""
|
| 32 |
+
# Color profile
|
| 33 |
+
mean_lab: np.ndarray = field(default_factory=lambda: np.zeros(3))
|
| 34 |
+
std_lab: np.ndarray = field(default_factory=lambda: np.ones(3))
|
| 35 |
+
melanin_level: float = 0.5 # 0.0=very fair, 1.0=very deep
|
| 36 |
+
undertone: str = "neutral" # warm, cool, neutral, olive
|
| 37 |
+
|
| 38 |
+
# Texture profile
|
| 39 |
+
pore_density: float = 0.5 # detected pore density
|
| 40 |
+
texture_roughness: float = 0.5 # skin surface roughness
|
| 41 |
+
micro_texture_map: Optional[np.ndarray] = None # high-freq detail map
|
| 42 |
+
|
| 43 |
+
# Imperfections (identity-critical — must be preserved)
|
| 44 |
+
freckle_map: Optional[np.ndarray] = None
|
| 45 |
+
mole_positions: list = field(default_factory=list)
|
| 46 |
+
beauty_marks: list = field(default_factory=list)
|
| 47 |
+
|
| 48 |
+
# Specular profile
|
| 49 |
+
specular_intensity: float = 0.3
|
| 50 |
+
specular_spread: float = 0.5
|
| 51 |
+
oiliness: float = 0.3 # T-zone shine level
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
class SkinRealismAgent:
|
| 55 |
+
"""
|
| 56 |
+
Skill agent for photorealistic skin texture in every generated frame.
|
| 57 |
+
|
| 58 |
+
Pipeline:
|
| 59 |
+
1. Analyze reference portrait → build SkinProfile
|
| 60 |
+
2. On each generated frame:
|
| 61 |
+
a. Extract face region
|
| 62 |
+
b. Apply melanin-aware color correction
|
| 63 |
+
c. Synthesize micro-texture (pores, fine lines)
|
| 64 |
+
d. Apply subsurface scattering simulation
|
| 65 |
+
e. Preserve identity markers (freckles, moles)
|
| 66 |
+
f. Add natural specular highlights
|
| 67 |
+
g. Apply dynamic skin response (emotion-driven)
|
| 68 |
+
3. Validate against Eden Protocol
|
| 69 |
+
"""
|
| 70 |
+
|
| 71 |
+
def __init__(self):
|
| 72 |
+
self.profile: Optional[SkinProfile] = None
|
| 73 |
+
self._reference_face: Optional[np.ndarray] = None
|
| 74 |
+
self._face_mask: Optional[np.ndarray] = None
|
| 75 |
+
self._initialized = False
|
| 76 |
+
|
| 77 |
+
# Tunable parameters (connected to admin panel sliders)
|
| 78 |
+
self.realism_strength = 0.7 # 0=none, 1=max processing
|
| 79 |
+
self.texture_detail = 0.6 # pore/texture enhancement level
|
| 80 |
+
self.sss_strength = 0.4 # subsurface scattering intensity
|
| 81 |
+
self.specular_strength = 0.3 # highlight intensity
|
| 82 |
+
self.imperfection_preserve = 0.9 # how strongly to keep freckles/moles
|
| 83 |
+
|
| 84 |
+
async def analyze_portrait(self, portrait: np.ndarray, face_bbox: Optional[tuple] = None) -> SkinProfile:
|
| 85 |
+
"""
|
| 86 |
+
Build a complete skin profile from the reference portrait.
|
| 87 |
+
This runs once during session setup (Genesis phase).
|
| 88 |
+
"""
|
| 89 |
+
logger.info("Skin Realism Agent: Analyzing reference portrait...")
|
| 90 |
+
|
| 91 |
+
profile = SkinProfile()
|
| 92 |
+
|
| 93 |
+
# Extract face region
|
| 94 |
+
if face_bbox:
|
| 95 |
+
x, y, w, h = face_bbox
|
| 96 |
+
face = portrait[y:y+h, x:x+w]
|
| 97 |
+
else:
|
| 98 |
+
face = self._detect_and_crop_face(portrait)
|
| 99 |
+
|
| 100 |
+
self._reference_face = face.copy()
|
| 101 |
+
h, w = face.shape[:2]
|
| 102 |
+
|
| 103 |
+
# Create skin mask (exclude eyes, lips, eyebrows, hair)
|
| 104 |
+
skin_mask = self._create_skin_mask(face)
|
| 105 |
+
self._face_mask = skin_mask
|
| 106 |
+
|
| 107 |
+
# ── Color Analysis ──
|
| 108 |
+
lab = cv2.cvtColor(face, cv2.COLOR_RGB2LAB).astype(np.float32)
|
| 109 |
+
skin_pixels = lab[skin_mask > 0]
|
| 110 |
+
|
| 111 |
+
if len(skin_pixels) > 0:
|
| 112 |
+
profile.mean_lab = np.mean(skin_pixels, axis=0)
|
| 113 |
+
profile.std_lab = np.std(skin_pixels, axis=0)
|
| 114 |
+
|
| 115 |
+
# Melanin estimation from L channel (luminance)
|
| 116 |
+
l_mean = profile.mean_lab[0]
|
| 117 |
+
# L ranges 0-255 in OpenCV LAB; darker skin = lower L
|
| 118 |
+
profile.melanin_level = np.clip(1.0 - (l_mean / 200.0), 0.0, 1.0)
|
| 119 |
+
|
| 120 |
+
# Undertone from a/b channels
|
| 121 |
+
a_mean = profile.mean_lab[1] - 128 # center around 0
|
| 122 |
+
b_mean = profile.mean_lab[2] - 128
|
| 123 |
+
if b_mean > 5 and a_mean > 0:
|
| 124 |
+
profile.undertone = "warm"
|
| 125 |
+
elif b_mean < -3:
|
| 126 |
+
profile.undertone = "cool"
|
| 127 |
+
elif a_mean > 3 and b_mean < 3:
|
| 128 |
+
profile.undertone = "olive"
|
| 129 |
+
else:
|
| 130 |
+
profile.undertone = "neutral"
|
| 131 |
+
|
| 132 |
+
# ── Texture Analysis ──
|
| 133 |
+
gray = cv2.cvtColor(face, cv2.COLOR_RGB2GRAY)
|
| 134 |
+
|
| 135 |
+
# Pore detection via Laplacian response
|
| 136 |
+
laplacian = cv2.Laplacian(gray, cv2.CV_64F)
|
| 137 |
+
pore_response = np.abs(laplacian)
|
| 138 |
+
skin_pore_response = pore_response[skin_mask > 0]
|
| 139 |
+
if len(skin_pore_response) > 0:
|
| 140 |
+
profile.pore_density = np.clip(np.mean(skin_pore_response) / 30.0, 0.0, 1.0)
|
| 141 |
+
|
| 142 |
+
# Surface roughness via Gabor energy
|
| 143 |
+
roughness_energies = []
|
| 144 |
+
for theta in [0, np.pi/4, np.pi/2, 3*np.pi/4]:
|
| 145 |
+
kernel = cv2.getGaborKernel((21, 21), 3.0, theta, 8.0, 0.5, 0)
|
| 146 |
+
response = cv2.filter2D(gray, cv2.CV_64F, kernel)
|
| 147 |
+
skin_response = response[skin_mask > 0]
|
| 148 |
+
if len(skin_response) > 0:
|
| 149 |
+
roughness_energies.append(np.std(skin_response))
|
| 150 |
+
|
| 151 |
+
if roughness_energies:
|
| 152 |
+
profile.texture_roughness = np.clip(np.mean(roughness_energies) / 50.0, 0.0, 1.0)
|
| 153 |
+
|
| 154 |
+
# Extract micro-texture map (high-frequency detail)
|
| 155 |
+
blurred = cv2.GaussianBlur(gray, (15, 15), 0)
|
| 156 |
+
profile.micro_texture_map = (gray.astype(np.float32) - blurred.astype(np.float32))
|
| 157 |
+
|
| 158 |
+
# ── Imperfection Detection ──
|
| 159 |
+
profile.freckle_map = self._detect_freckles(face, skin_mask)
|
| 160 |
+
profile.mole_positions = self._detect_moles(face, skin_mask)
|
| 161 |
+
|
| 162 |
+
# ── Specular Profile ──
|
| 163 |
+
# Detect existing highlights (bright spots on skin)
|
| 164 |
+
hsv = cv2.cvtColor(face, cv2.COLOR_RGB2HSV)
|
| 165 |
+
v_channel = hsv[:, :, 2].astype(np.float32)
|
| 166 |
+
skin_brightness = v_channel[skin_mask > 0]
|
| 167 |
+
if len(skin_brightness) > 0:
|
| 168 |
+
bright_threshold = np.percentile(skin_brightness, 95)
|
| 169 |
+
highlight_ratio = np.mean(skin_brightness > bright_threshold)
|
| 170 |
+
profile.specular_intensity = np.clip(highlight_ratio * 10, 0.1, 0.8)
|
| 171 |
+
|
| 172 |
+
# T-zone oiliness (forehead + nose region)
|
| 173 |
+
t_zone_y = slice(0, h // 3)
|
| 174 |
+
t_zone_x = slice(w // 4, 3 * w // 4)
|
| 175 |
+
t_zone_v = v_channel[t_zone_y, t_zone_x]
|
| 176 |
+
t_zone_mask = skin_mask[t_zone_y, t_zone_x]
|
| 177 |
+
t_zone_skin = t_zone_v[t_zone_mask > 0]
|
| 178 |
+
if len(t_zone_skin) > 0:
|
| 179 |
+
profile.oiliness = np.clip(
|
| 180 |
+
(np.mean(t_zone_skin) - np.mean(skin_brightness)) / 30.0 + 0.3,
|
| 181 |
+
0.0, 1.0
|
| 182 |
+
)
|
| 183 |
+
|
| 184 |
+
self.profile = profile
|
| 185 |
+
self._initialized = True
|
| 186 |
+
|
| 187 |
+
logger.info(
|
| 188 |
+
f"Skin profile built: melanin={profile.melanin_level:.2f}, "
|
| 189 |
+
f"undertone={profile.undertone}, pores={profile.pore_density:.2f}, "
|
| 190 |
+
f"roughness={profile.texture_roughness:.2f}, "
|
| 191 |
+
f"moles={len(profile.mole_positions)}"
|
| 192 |
+
)
|
| 193 |
+
|
| 194 |
+
return profile
|
| 195 |
+
|
| 196 |
+
def enhance_frame(
|
| 197 |
+
self,
|
| 198 |
+
frame: np.ndarray,
|
| 199 |
+
face_bbox: Optional[tuple] = None,
|
| 200 |
+
emotion: Optional[dict] = None,
|
| 201 |
+
) -> np.ndarray:
|
| 202 |
+
"""
|
| 203 |
+
Apply skin realism enhancement to a generated frame.
|
| 204 |
+
Called on every frame from the Animator pipeline.
|
| 205 |
+
"""
|
| 206 |
+
if not self._initialized or self.profile is None:
|
| 207 |
+
return frame
|
| 208 |
+
|
| 209 |
+
if self.realism_strength < 0.01:
|
| 210 |
+
return frame
|
| 211 |
+
|
| 212 |
+
result = frame.copy()
|
| 213 |
+
|
| 214 |
+
# Extract face region
|
| 215 |
+
if face_bbox:
|
| 216 |
+
x, y, w, h = face_bbox
|
| 217 |
+
face = result[y:y+h, x:x+w]
|
| 218 |
+
else:
|
| 219 |
+
face = result # assume face-cropped input
|
| 220 |
+
|
| 221 |
+
# Step 1: Melanin-aware color correction
|
| 222 |
+
face = self._apply_color_correction(face)
|
| 223 |
+
|
| 224 |
+
# Step 2: Micro-texture synthesis
|
| 225 |
+
if self.texture_detail > 0.1:
|
| 226 |
+
face = self._apply_micro_texture(face)
|
| 227 |
+
|
| 228 |
+
# Step 3: Subsurface scattering simulation
|
| 229 |
+
if self.sss_strength > 0.1:
|
| 230 |
+
face = self._apply_sss(face)
|
| 231 |
+
|
| 232 |
+
# Step 4: Preserve identity markers
|
| 233 |
+
if self.imperfection_preserve > 0.1:
|
| 234 |
+
face = self._preserve_imperfections(face)
|
| 235 |
+
|
| 236 |
+
# Step 5: Natural specular highlights
|
| 237 |
+
if self.specular_strength > 0.1:
|
| 238 |
+
face = self._apply_specular(face)
|
| 239 |
+
|
| 240 |
+
# Step 6: Dynamic skin response (emotion-driven)
|
| 241 |
+
if emotion:
|
| 242 |
+
face = self._apply_dynamic_response(face, emotion)
|
| 243 |
+
|
| 244 |
+
# Write back to frame
|
| 245 |
+
if face_bbox:
|
| 246 |
+
result[y:y+h, x:x+w] = face
|
| 247 |
+
else:
|
| 248 |
+
result = face
|
| 249 |
+
|
| 250 |
+
return result
|
| 251 |
+
|
| 252 |
+
# ── Processing Steps ───────────────────────────────────────────
|
| 253 |
+
|
| 254 |
+
def _apply_color_correction(self, face: np.ndarray) -> np.ndarray:
|
| 255 |
+
"""
|
| 256 |
+
Melanin-aware color correction.
|
| 257 |
+
Prevents the "whitewashing" artifact common in AI face generation.
|
| 258 |
+
Ensures generated skin tone matches reference across all melanin levels.
|
| 259 |
+
"""
|
| 260 |
+
if self.profile is None:
|
| 261 |
+
return face
|
| 262 |
+
|
| 263 |
+
lab = cv2.cvtColor(face, cv2.COLOR_RGB2LAB).astype(np.float32)
|
| 264 |
+
|
| 265 |
+
# Get current skin color stats
|
| 266 |
+
mask = self._create_skin_mask(face) if self._face_mask is None else cv2.resize(
|
| 267 |
+
self._face_mask, (face.shape[1], face.shape[0])
|
| 268 |
+
)
|
| 269 |
+
current_pixels = lab[mask > 0]
|
| 270 |
+
|
| 271 |
+
if len(current_pixels) < 100:
|
| 272 |
+
return face
|
| 273 |
+
|
| 274 |
+
current_mean = np.mean(current_pixels, axis=0)
|
| 275 |
+
|
| 276 |
+
# Compute correction needed
|
| 277 |
+
target_mean = self.profile.mean_lab
|
| 278 |
+
correction = (target_mean - current_mean) * self.realism_strength * 0.5
|
| 279 |
+
|
| 280 |
+
# Apply correction more aggressively on color channels (a, b)
|
| 281 |
+
# and more gently on luminance (L) to preserve lighting
|
| 282 |
+
lab[:, :, 0] += correction[0] * 0.3 # gentle L correction
|
| 283 |
+
lab[:, :, 1] += correction[1] * 0.7 # stronger a correction
|
| 284 |
+
lab[:, :, 2] += correction[2] * 0.7 # stronger b correction
|
| 285 |
+
|
| 286 |
+
lab = np.clip(lab, 0, 255)
|
| 287 |
+
corrected = cv2.cvtColor(lab.astype(np.uint8), cv2.COLOR_LAB2RGB)
|
| 288 |
+
|
| 289 |
+
# Blend with mask (only apply to skin, not eyes/lips)
|
| 290 |
+
mask_3ch = np.stack([mask, mask, mask], axis=-1).astype(np.float32) / 255.0
|
| 291 |
+
blended = (corrected.astype(np.float32) * mask_3ch +
|
| 292 |
+
face.astype(np.float32) * (1 - mask_3ch))
|
| 293 |
+
|
| 294 |
+
return np.clip(blended, 0, 255).astype(np.uint8)
|
| 295 |
+
|
| 296 |
+
def _apply_micro_texture(self, face: np.ndarray) -> np.ndarray:
|
| 297 |
+
"""
|
| 298 |
+
Synthesize and overlay pore-level micro-texture.
|
| 299 |
+
Uses the reference portrait's texture map to add authentic skin detail.
|
| 300 |
+
"""
|
| 301 |
+
if self.profile is None or self.profile.micro_texture_map is None:
|
| 302 |
+
return face
|
| 303 |
+
|
| 304 |
+
h, w = face.shape[:2]
|
| 305 |
+
texture_map = cv2.resize(
|
| 306 |
+
self.profile.micro_texture_map, (w, h)
|
| 307 |
+
).astype(np.float32)
|
| 308 |
+
|
| 309 |
+
# Scale texture based on target pore density
|
| 310 |
+
texture_strength = self.texture_detail * self.profile.pore_density * 0.4
|
| 311 |
+
|
| 312 |
+
# Add high-frequency noise for organic randomness
|
| 313 |
+
noise = np.random.normal(0, 1, (h, w)).astype(np.float32) * 0.5
|
| 314 |
+
organic_texture = texture_map + noise * texture_strength
|
| 315 |
+
|
| 316 |
+
# Apply texture overlay (add to luminance channel)
|
| 317 |
+
lab = cv2.cvtColor(face, cv2.COLOR_RGB2LAB).astype(np.float32)
|
| 318 |
+
lab[:, :, 0] += organic_texture * texture_strength * 15
|
| 319 |
+
lab[:, :, 0] = np.clip(lab[:, :, 0], 0, 255)
|
| 320 |
+
|
| 321 |
+
return cv2.cvtColor(lab.astype(np.uint8), cv2.COLOR_LAB2RGB)
|
| 322 |
+
|
| 323 |
+
def _apply_sss(self, face: np.ndarray) -> np.ndarray:
|
| 324 |
+
"""
|
| 325 |
+
Subsurface Scattering (SSS) simulation.
|
| 326 |
+
Real skin is translucent — light penetrates and scatters under the surface,
|
| 327 |
+
creating a warm glow especially at edges (ears, nose, thin skin areas).
|
| 328 |
+
AI-generated faces lack this, making them look "plastic."
|
| 329 |
+
"""
|
| 330 |
+
h, w = face.shape[:2]
|
| 331 |
+
|
| 332 |
+
# SSS effect: blur the red channel more than green/blue
|
| 333 |
+
# (blood absorbs blue/green, transmits red → warm subsurface glow)
|
| 334 |
+
r, g, b = face[:, :, 0], face[:, :, 1], face[:, :, 2]
|
| 335 |
+
|
| 336 |
+
# Different blur radii per channel simulate wavelength-dependent scattering
|
| 337 |
+
blur_radius = max(3, int(min(h, w) * 0.02))
|
| 338 |
+
if blur_radius % 2 == 0:
|
| 339 |
+
blur_radius += 1
|
| 340 |
+
|
| 341 |
+
r_k = blur_radius * 3 | 1 # ensure odd
|
| 342 |
+
g_k = blur_radius * 2 | 1
|
| 343 |
+
b_k = blur_radius | 1
|
| 344 |
+
r_sss = cv2.GaussianBlur(r, (r_k, r_k), 0)
|
| 345 |
+
g_sss = cv2.GaussianBlur(g, (g_k, g_k), 0)
|
| 346 |
+
b_sss = cv2.GaussianBlur(b, (b_k, b_k), 0)
|
| 347 |
+
|
| 348 |
+
# Blend SSS with original
|
| 349 |
+
strength = self.sss_strength * self.realism_strength
|
| 350 |
+
|
| 351 |
+
# Melanin-aware: darker skin has less visible SSS
|
| 352 |
+
melanin_factor = 1.0 - (self.profile.melanin_level * 0.5) if self.profile else 0.7
|
| 353 |
+
strength *= melanin_factor
|
| 354 |
+
|
| 355 |
+
result = face.copy().astype(np.float32)
|
| 356 |
+
result[:, :, 0] = r * (1 - strength * 0.15) + r_sss * (strength * 0.15)
|
| 357 |
+
result[:, :, 1] = g * (1 - strength * 0.08) + g_sss * (strength * 0.08)
|
| 358 |
+
result[:, :, 2] = b * (1 - strength * 0.03) + b_sss * (strength * 0.03)
|
| 359 |
+
|
| 360 |
+
return np.clip(result, 0, 255).astype(np.uint8)
|
| 361 |
+
|
| 362 |
+
def _preserve_imperfections(self, face: np.ndarray) -> np.ndarray:
|
| 363 |
+
"""
|
| 364 |
+
Re-inject identity-critical skin imperfections.
|
| 365 |
+
Freckles, moles, and beauty marks must survive the generation process.
|
| 366 |
+
"""
|
| 367 |
+
if self.profile is None:
|
| 368 |
+
return face
|
| 369 |
+
|
| 370 |
+
h, w = face.shape[:2]
|
| 371 |
+
strength = self.imperfection_preserve * self.realism_strength
|
| 372 |
+
|
| 373 |
+
# Overlay freckle map
|
| 374 |
+
if self.profile.freckle_map is not None and strength > 0.3:
|
| 375 |
+
freckle_resized = cv2.resize(self.profile.freckle_map, (w, h))
|
| 376 |
+
# Freckles darken the skin slightly
|
| 377 |
+
freckle_mask = (freckle_resized > 30).astype(np.float32)
|
| 378 |
+
darkening = freckle_resized.astype(np.float32) * strength * 0.3
|
| 379 |
+
|
| 380 |
+
lab = cv2.cvtColor(face, cv2.COLOR_RGB2LAB).astype(np.float32)
|
| 381 |
+
lab[:, :, 0] -= darkening * freckle_mask
|
| 382 |
+
lab[:, :, 0] = np.clip(lab[:, :, 0], 0, 255)
|
| 383 |
+
face = cv2.cvtColor(lab.astype(np.uint8), cv2.COLOR_LAB2RGB)
|
| 384 |
+
|
| 385 |
+
# Re-stamp moles at stored positions
|
| 386 |
+
for mole in self.profile.mole_positions:
|
| 387 |
+
mx = int(mole["x"] * w)
|
| 388 |
+
my = int(mole["y"] * h)
|
| 389 |
+
radius = max(1, int(mole.get("radius", 2) * min(w, h) / 512))
|
| 390 |
+
|
| 391 |
+
if 0 <= mx < w and 0 <= my < h:
|
| 392 |
+
# Darken a small circular region
|
| 393 |
+
cv2.circle(face, (mx, my), radius,
|
| 394 |
+
(int(mole.get("r", 80)), int(mole.get("g", 60)), int(mole.get("b", 50))),
|
| 395 |
+
-1)
|
| 396 |
+
|
| 397 |
+
return face
|
| 398 |
+
|
| 399 |
+
def _apply_specular(self, face: np.ndarray) -> np.ndarray:
|
| 400 |
+
"""
|
| 401 |
+
Add natural specular highlights.
|
| 402 |
+
Real skin has a micro-sheen from oil/moisture — not the uniform matte
|
| 403 |
+
look of AI-generated faces, nor the plastic-looking spec of bad CGI.
|
| 404 |
+
"""
|
| 405 |
+
h, w = face.shape[:2]
|
| 406 |
+
strength = self.specular_strength * self.realism_strength
|
| 407 |
+
|
| 408 |
+
if self.profile is None:
|
| 409 |
+
return face
|
| 410 |
+
|
| 411 |
+
# Create specular highlight map based on face geometry
|
| 412 |
+
# Highlights concentrate on: nose bridge, forehead, cheekbones, chin
|
| 413 |
+
highlight_map = np.zeros((h, w), dtype=np.float32)
|
| 414 |
+
|
| 415 |
+
# Nose bridge (center top)
|
| 416 |
+
cv2.ellipse(highlight_map, (w // 2, int(h * 0.45)),
|
| 417 |
+
(w // 12, h // 6), 0, 0, 360, 1.0, -1)
|
| 418 |
+
|
| 419 |
+
# Forehead
|
| 420 |
+
cv2.ellipse(highlight_map, (w // 2, int(h * 0.2)),
|
| 421 |
+
(w // 4, h // 8), 0, 0, 360, 0.6, -1)
|
| 422 |
+
|
| 423 |
+
# Cheekbones (left and right)
|
| 424 |
+
cv2.ellipse(highlight_map, (int(w * 0.3), int(h * 0.5)),
|
| 425 |
+
(w // 8, h // 10), 15, 0, 360, 0.4, -1)
|
| 426 |
+
cv2.ellipse(highlight_map, (int(w * 0.7), int(h * 0.5)),
|
| 427 |
+
(w // 8, h // 10), -15, 0, 360, 0.4, -1)
|
| 428 |
+
|
| 429 |
+
# Blur for smooth falloff
|
| 430 |
+
highlight_map = cv2.GaussianBlur(highlight_map, (31, 31), 0)
|
| 431 |
+
highlight_map *= strength * self.profile.specular_intensity
|
| 432 |
+
|
| 433 |
+
# Oiliness increases T-zone highlights
|
| 434 |
+
if self.profile.oiliness > 0.3:
|
| 435 |
+
t_zone = np.zeros((h, w), dtype=np.float32)
|
| 436 |
+
cv2.ellipse(t_zone, (w // 2, int(h * 0.3)),
|
| 437 |
+
(w // 6, h // 4), 0, 0, 360, self.profile.oiliness * 0.3, -1)
|
| 438 |
+
t_zone = cv2.GaussianBlur(t_zone, (21, 21), 0)
|
| 439 |
+
highlight_map += t_zone
|
| 440 |
+
|
| 441 |
+
# Apply highlights (additive on luminance)
|
| 442 |
+
lab = cv2.cvtColor(face, cv2.COLOR_RGB2LAB).astype(np.float32)
|
| 443 |
+
lab[:, :, 0] += highlight_map * 30
|
| 444 |
+
lab[:, :, 0] = np.clip(lab[:, :, 0], 0, 255)
|
| 445 |
+
|
| 446 |
+
return cv2.cvtColor(lab.astype(np.uint8), cv2.COLOR_LAB2RGB)
|
| 447 |
+
|
| 448 |
+
def _apply_dynamic_response(self, face: np.ndarray, emotion: dict) -> np.ndarray:
|
| 449 |
+
"""
|
| 450 |
+
Emotion-driven skin response.
|
| 451 |
+
- Joy/warmth → slight blush on cheeks
|
| 452 |
+
- Fear/urgency → slight pallor
|
| 453 |
+
- Confidence → healthy glow
|
| 454 |
+
- Sadness → reduced color saturation
|
| 455 |
+
"""
|
| 456 |
+
h, w = face.shape[:2]
|
| 457 |
+
lab = cv2.cvtColor(face, cv2.COLOR_RGB2LAB).astype(np.float32)
|
| 458 |
+
|
| 459 |
+
joy = emotion.get("joy", 0.5)
|
| 460 |
+
warmth = emotion.get("warmth", 0.5)
|
| 461 |
+
sadness = emotion.get("sadness", 0.0)
|
| 462 |
+
urgency = emotion.get("urgency", 0.0)
|
| 463 |
+
|
| 464 |
+
# Blush response (joy + warmth increase redness on cheeks)
|
| 465 |
+
blush_intensity = max(0, (joy - 0.5) + (warmth - 0.5)) * 0.3
|
| 466 |
+
if blush_intensity > 0.05:
|
| 467 |
+
blush_map = np.zeros((h, w), dtype=np.float32)
|
| 468 |
+
# Cheek regions
|
| 469 |
+
cv2.ellipse(blush_map, (int(w * 0.3), int(h * 0.6)),
|
| 470 |
+
(w // 6, h // 8), 0, 0, 360, blush_intensity, -1)
|
| 471 |
+
cv2.ellipse(blush_map, (int(w * 0.7), int(h * 0.6)),
|
| 472 |
+
(w // 6, h // 8), 0, 0, 360, blush_intensity, -1)
|
| 473 |
+
blush_map = cv2.GaussianBlur(blush_map, (31, 31), 0)
|
| 474 |
+
|
| 475 |
+
# Increase a-channel (red-green axis in LAB) for blush
|
| 476 |
+
lab[:, :, 1] += blush_map * 15
|
| 477 |
+
|
| 478 |
+
# Pallor response (urgency/fear reduces color)
|
| 479 |
+
if urgency > 0.5:
|
| 480 |
+
pallor = (urgency - 0.5) * 0.2
|
| 481 |
+
lab[:, :, 0] += pallor * 10 # slightly brighter (pale)
|
| 482 |
+
lab[:, :, 1] -= pallor * 5 # less red
|
| 483 |
+
lab[:, :, 2] -= pallor * 3 # less yellow
|
| 484 |
+
|
| 485 |
+
# Sadness: desaturate slightly
|
| 486 |
+
if sadness > 0.3:
|
| 487 |
+
desat = (sadness - 0.3) * 0.15
|
| 488 |
+
lab[:, :, 1] = lab[:, :, 1] * (1 - desat) + 128 * desat
|
| 489 |
+
lab[:, :, 2] = lab[:, :, 2] * (1 - desat) + 128 * desat
|
| 490 |
+
|
| 491 |
+
lab = np.clip(lab, 0, 255)
|
| 492 |
+
return cv2.cvtColor(lab.astype(np.uint8), cv2.COLOR_LAB2RGB)
|
| 493 |
+
|
| 494 |
+
# ── Helper Methods ─────────────────────────────────────────────
|
| 495 |
+
|
| 496 |
+
def _detect_and_crop_face(self, image: np.ndarray) -> np.ndarray:
|
| 497 |
+
"""Simple face detection and crop."""
|
| 498 |
+
try:
|
| 499 |
+
import mediapipe as mp
|
| 500 |
+
detector = mp.solutions.face_detection.FaceDetection(min_detection_confidence=0.5)
|
| 501 |
+
results = detector.process(image)
|
| 502 |
+
if results.detections:
|
| 503 |
+
bbox = results.detections[0].location_data.relative_bounding_box
|
| 504 |
+
h, w = image.shape[:2]
|
| 505 |
+
x = max(0, int(bbox.xmin * w))
|
| 506 |
+
y = max(0, int(bbox.ymin * h))
|
| 507 |
+
fw = min(w - x, int(bbox.width * w))
|
| 508 |
+
fh = min(h - y, int(bbox.height * h))
|
| 509 |
+
return image[y:y+fh, x:x+fw]
|
| 510 |
+
except Exception:
|
| 511 |
+
pass
|
| 512 |
+
return image
|
| 513 |
+
|
| 514 |
+
def _create_skin_mask(self, face: np.ndarray) -> np.ndarray:
|
| 515 |
+
"""Create binary mask of skin pixels using HSV thresholding."""
|
| 516 |
+
hsv = cv2.cvtColor(face, cv2.COLOR_RGB2HSV)
|
| 517 |
+
|
| 518 |
+
# Broad skin color range in HSV
|
| 519 |
+
lower = np.array([0, 20, 50], dtype=np.uint8)
|
| 520 |
+
upper = np.array([35, 255, 255], dtype=np.uint8)
|
| 521 |
+
mask1 = cv2.inRange(hsv, lower, upper)
|
| 522 |
+
|
| 523 |
+
# Additional range for darker skin tones
|
| 524 |
+
lower2 = np.array([0, 10, 30], dtype=np.uint8)
|
| 525 |
+
upper2 = np.array([20, 200, 200], dtype=np.uint8)
|
| 526 |
+
mask2 = cv2.inRange(hsv, lower2, upper2)
|
| 527 |
+
|
| 528 |
+
mask = cv2.bitwise_or(mask1, mask2)
|
| 529 |
+
|
| 530 |
+
# Clean up with morphology
|
| 531 |
+
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
|
| 532 |
+
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
|
| 533 |
+
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
|
| 534 |
+
|
| 535 |
+
return mask
|
| 536 |
+
|
| 537 |
+
def _detect_freckles(self, face: np.ndarray, skin_mask: np.ndarray) -> np.ndarray:
|
| 538 |
+
"""Detect freckle pattern from reference portrait."""
|
| 539 |
+
gray = cv2.cvtColor(face, cv2.COLOR_RGB2GRAY)
|
| 540 |
+
|
| 541 |
+
# Freckles are small dark spots on lighter skin
|
| 542 |
+
blurred = cv2.GaussianBlur(gray, (11, 11), 0)
|
| 543 |
+
diff = blurred.astype(np.float32) - gray.astype(np.float32)
|
| 544 |
+
|
| 545 |
+
# Freckles are where the original is darker than the blur
|
| 546 |
+
freckle_map = np.clip(diff, 0, 255).astype(np.uint8)
|
| 547 |
+
|
| 548 |
+
# Only on skin
|
| 549 |
+
freckle_map = cv2.bitwise_and(freckle_map, freckle_map, mask=skin_mask)
|
| 550 |
+
|
| 551 |
+
# Threshold to get significant spots only
|
| 552 |
+
_, freckle_map = cv2.threshold(freckle_map, 15, 255, cv2.THRESH_TOZERO)
|
| 553 |
+
|
| 554 |
+
return freckle_map
|
| 555 |
+
|
| 556 |
+
def _detect_moles(self, face: np.ndarray, skin_mask: np.ndarray) -> list:
|
| 557 |
+
"""Detect moles and beauty marks from reference portrait."""
|
| 558 |
+
gray = cv2.cvtColor(face, cv2.COLOR_RGB2GRAY)
|
| 559 |
+
h, w = face.shape[:2]
|
| 560 |
+
|
| 561 |
+
# Moles are dark, round, small features
|
| 562 |
+
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
|
| 563 |
+
|
| 564 |
+
# Adaptive threshold to find dark spots
|
| 565 |
+
thresh = cv2.adaptiveThreshold(
|
| 566 |
+
blurred, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
|
| 567 |
+
cv2.THRESH_BINARY_INV, 11, 8
|
| 568 |
+
)
|
| 569 |
+
|
| 570 |
+
# Only on skin
|
| 571 |
+
thresh = cv2.bitwise_and(thresh, thresh, mask=skin_mask)
|
| 572 |
+
|
| 573 |
+
# Find contours
|
| 574 |
+
contours, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
|
| 575 |
+
|
| 576 |
+
moles = []
|
| 577 |
+
for contour in contours:
|
| 578 |
+
area = cv2.contourArea(contour)
|
| 579 |
+
# Moles are small (3-30 pixels area at 512x512)
|
| 580 |
+
if 3 < area < 50:
|
| 581 |
+
(cx, cy), radius = cv2.minEnclosingCircle(contour)
|
| 582 |
+
# Check circularity
|
| 583 |
+
perimeter = cv2.arcLength(contour, True)
|
| 584 |
+
if perimeter > 0:
|
| 585 |
+
circularity = 4 * np.pi * area / (perimeter ** 2)
|
| 586 |
+
if circularity > 0.4: # reasonably round
|
| 587 |
+
# Get color at mole position
|
| 588 |
+
mx, my = int(cx), int(cy)
|
| 589 |
+
if 0 <= mx < w and 0 <= my < h:
|
| 590 |
+
color = face[my, mx]
|
| 591 |
+
moles.append({
|
| 592 |
+
"x": cx / w, # normalized position
|
| 593 |
+
"y": cy / h,
|
| 594 |
+
"radius": radius,
|
| 595 |
+
"r": int(color[0]),
|
| 596 |
+
"g": int(color[1]),
|
| 597 |
+
"b": int(color[2]),
|
| 598 |
+
})
|
| 599 |
+
|
| 600 |
+
logger.debug(f"Detected {len(moles)} moles/beauty marks")
|
| 601 |
+
return moles[:20] # Cap at 20 to avoid false positives
|
| 602 |
+
|
| 603 |
+
def update_settings(self, settings: dict) -> None:
|
| 604 |
+
"""Update from admin panel sliders."""
|
| 605 |
+
if "skin_realism" in settings:
|
| 606 |
+
self.realism_strength = float(settings["skin_realism"])
|
| 607 |
+
if "texture_detail" in settings:
|
| 608 |
+
self.texture_detail = float(settings["texture_detail"])
|
| 609 |
+
if "sss_strength" in settings:
|
| 610 |
+
self.sss_strength = float(settings["sss_strength"])
|
| 611 |
+
if "specular" in settings:
|
| 612 |
+
self.specular_strength = float(settings["specular"])
|
| 613 |
+
|
| 614 |
+
def get_status(self) -> dict:
|
| 615 |
+
"""Get agent status."""
|
| 616 |
+
return {
|
| 617 |
+
"initialized": self._initialized,
|
| 618 |
+
"melanin_level": self.profile.melanin_level if self.profile else None,
|
| 619 |
+
"undertone": self.profile.undertone if self.profile else None,
|
| 620 |
+
"pore_density": self.profile.pore_density if self.profile else None,
|
| 621 |
+
"moles_detected": len(self.profile.mole_positions) if self.profile else 0,
|
| 622 |
+
"realism_strength": self.realism_strength,
|
| 623 |
+
}
|
scripts/save_versioned.sh
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
# ═══════════════════════════════════════════════════════════════
|
| 3 |
+
# EDEN OS — Versioned Save Protocol
|
| 4 |
+
# Saves to: GitHub, HuggingFace, Seagate 5TB (with version tags)
|
| 5 |
+
#
|
| 6 |
+
# Usage: bash scripts/save_versioned.sh [version_tag]
|
| 7 |
+
# Example: bash scripts/save_versioned.sh v1.0.1
|
| 8 |
+
# ═══════════════════════════════════════════════════════════════
|
| 9 |
+
|
| 10 |
+
set -e
|
| 11 |
+
cd "$(dirname "$0")/.."
|
| 12 |
+
|
| 13 |
+
VERSION="${1:-$(date +v%Y%m%d_%H%M%S)}"
|
| 14 |
+
TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")
|
| 15 |
+
|
| 16 |
+
echo "═══════════════════════════════════════════════════"
|
| 17 |
+
echo " EDEN OS — VERSIONED SAVE: $VERSION"
|
| 18 |
+
echo " $TIMESTAMP"
|
| 19 |
+
echo "═══════════════════════════════════════════════════"
|
| 20 |
+
|
| 21 |
+
# ── 1. Git Commit + Tag ──────────────────────────────────────
|
| 22 |
+
echo ""
|
| 23 |
+
echo "[1/4] Git commit + tag..."
|
| 24 |
+
git add -A
|
| 25 |
+
git commit -m "EDEN OS $VERSION — $TIMESTAMP" --allow-empty 2>/dev/null || echo "Nothing to commit"
|
| 26 |
+
git tag -a "$VERSION" -m "EDEN OS $VERSION — $TIMESTAMP" 2>/dev/null || echo "Tag $VERSION already exists"
|
| 27 |
+
|
| 28 |
+
# ── 2. Push to GitHub ────────────────────────────────────────
|
| 29 |
+
echo ""
|
| 30 |
+
echo "[2/4] Pushing to GitHub (tyronne-os/EDEN-OS)..."
|
| 31 |
+
git push origin main --tags 2>&1 || echo "GitHub push failed (check connectivity)"
|
| 32 |
+
|
| 33 |
+
# ── 3. Push to HuggingFace ───────────────────────────────────
|
| 34 |
+
echo ""
|
| 35 |
+
echo "[3/4] Uploading to HuggingFace (AIBRUH/eden-os)..."
|
| 36 |
+
python3 -c "
|
| 37 |
+
from huggingface_hub import HfApi
|
| 38 |
+
api = HfApi()
|
| 39 |
+
api.upload_folder(
|
| 40 |
+
folder_path='.',
|
| 41 |
+
repo_id='AIBRUH/eden-os',
|
| 42 |
+
repo_type='model',
|
| 43 |
+
ignore_patterns=['.venv/*', '__pycache__/*', '*.pyc', 'models_cache/*', 'data/*', '.git/*'],
|
| 44 |
+
commit_message='EDEN OS $VERSION',
|
| 45 |
+
)
|
| 46 |
+
print('HuggingFace upload complete')
|
| 47 |
+
" 2>&1 || echo "HuggingFace upload failed"
|
| 48 |
+
|
| 49 |
+
# ── 4. Save to Seagate 5TB (versioned) ───────────────────────
|
| 50 |
+
echo ""
|
| 51 |
+
echo "[4/4] Saving to Seagate 5TB..."
|
| 52 |
+
|
| 53 |
+
SEAGATE_PATH=""
|
| 54 |
+
# Detect Seagate
|
| 55 |
+
for path in /mnt/s /mnt/S /mnt/seagate5tb; do
|
| 56 |
+
if [ -d "$path" ] && touch "$path/.eden_test" 2>/dev/null; then
|
| 57 |
+
rm "$path/.eden_test"
|
| 58 |
+
SEAGATE_PATH="$path"
|
| 59 |
+
break
|
| 60 |
+
fi
|
| 61 |
+
done
|
| 62 |
+
|
| 63 |
+
# Try PowerShell detection
|
| 64 |
+
if [ -z "$SEAGATE_PATH" ]; then
|
| 65 |
+
LETTER=$(powershell.exe -Command "(Get-Volume -FileSystemLabel 'SEAGATE5TB').DriveLetter" 2>/dev/null | tr -d '\r\n')
|
| 66 |
+
if [ -n "$LETTER" ]; then
|
| 67 |
+
WSL_PATH="/mnt/$(echo $LETTER | tr 'A-Z' 'a-z')"
|
| 68 |
+
if [ -d "$WSL_PATH" ]; then
|
| 69 |
+
SEAGATE_PATH="$WSL_PATH"
|
| 70 |
+
else
|
| 71 |
+
sudo mkdir -p /mnt/seagate5tb 2>/dev/null
|
| 72 |
+
sudo mount -t drvfs "${LETTER}:" /mnt/seagate5tb 2>/dev/null && SEAGATE_PATH="/mnt/seagate5tb"
|
| 73 |
+
fi
|
| 74 |
+
fi
|
| 75 |
+
fi
|
| 76 |
+
|
| 77 |
+
if [ -n "$SEAGATE_PATH" ]; then
|
| 78 |
+
BACKUP_DIR="$SEAGATE_PATH/eden-os/versions/$VERSION"
|
| 79 |
+
LATEST_DIR="$SEAGATE_PATH/eden-os/latest"
|
| 80 |
+
|
| 81 |
+
echo "Seagate detected at: $SEAGATE_PATH"
|
| 82 |
+
|
| 83 |
+
# Create versioned backup
|
| 84 |
+
mkdir -p "$BACKUP_DIR"
|
| 85 |
+
rsync -a --exclude='.venv' --exclude='__pycache__' --exclude='*.pyc' \
|
| 86 |
+
--exclude='models_cache' --exclude='data' --exclude='.git' \
|
| 87 |
+
. "$BACKUP_DIR/"
|
| 88 |
+
|
| 89 |
+
# Update latest symlink
|
| 90 |
+
rm -f "$LATEST_DIR" 2>/dev/null
|
| 91 |
+
ln -sf "$BACKUP_DIR" "$LATEST_DIR"
|
| 92 |
+
|
| 93 |
+
# Write version manifest
|
| 94 |
+
cat > "$SEAGATE_PATH/eden-os/VERSION_LOG.md" << MANIFEST
|
| 95 |
+
# EDEN OS — Version Log (Seagate 5TB)
|
| 96 |
+
Last updated: $TIMESTAMP
|
| 97 |
+
Latest version: $VERSION
|
| 98 |
+
|
| 99 |
+
## Versions on disk:
|
| 100 |
+
$(ls -1d "$SEAGATE_PATH/eden-os/versions/"* 2>/dev/null | while read d; do
|
| 101 |
+
V=$(basename "$d")
|
| 102 |
+
SIZE=$(du -sh "$d" 2>/dev/null | cut -f1)
|
| 103 |
+
echo "- $V ($SIZE)"
|
| 104 |
+
done)
|
| 105 |
+
MANIFEST
|
| 106 |
+
|
| 107 |
+
echo "Saved to Seagate: $BACKUP_DIR"
|
| 108 |
+
echo "Latest symlink: $LATEST_DIR"
|
| 109 |
+
|
| 110 |
+
# Cleanup: keep only last 10 versions
|
| 111 |
+
VERSIONS_DIR="$SEAGATE_PATH/eden-os/versions"
|
| 112 |
+
VERSION_COUNT=$(ls -1d "$VERSIONS_DIR/"* 2>/dev/null | wc -l)
|
| 113 |
+
if [ "$VERSION_COUNT" -gt 10 ]; then
|
| 114 |
+
echo "Cleaning old versions (keeping last 10)..."
|
| 115 |
+
ls -1td "$VERSIONS_DIR/"* | tail -n +11 | while read old; do
|
| 116 |
+
echo " Removing: $(basename $old)"
|
| 117 |
+
rm -rf "$old"
|
| 118 |
+
done
|
| 119 |
+
fi
|
| 120 |
+
else
|
| 121 |
+
echo "Seagate 5TB not connected — skipping local backup"
|
| 122 |
+
echo "Plug in Seagate and re-run: bash scripts/save_versioned.sh $VERSION"
|
| 123 |
+
fi
|
| 124 |
+
|
| 125 |
+
echo ""
|
| 126 |
+
echo "═══════════════════════════════════════════════════"
|
| 127 |
+
echo " EDEN OS $VERSION — SAVED"
|
| 128 |
+
echo " GitHub: tyronne-os/EDEN-OS"
|
| 129 |
+
echo " HuggingFace: AIBRUH/eden-os"
|
| 130 |
+
echo " Seagate: $([ -n "$SEAGATE_PATH" ] && echo "$SEAGATE_PATH/eden-os/versions/$VERSION" || echo "NOT CONNECTED")"
|
| 131 |
+
echo " OWN THE SCIENCE."
|
| 132 |
+
echo "═══════════════════════════════════════════════════"
|