Spaces:
Running on Zero
Running on Zero
Bill the conditioner call to the requesting user's own ZeroGPU token, never an org token
Browse files
README.md
CHANGED
|
@@ -160,31 +160,29 @@ one-time `PIPE.to("cuda")` is inside the first row's 339 s and does not reappear
|
|
| 160 |
|
| 161 |
## Whose GPU quota pays
|
| 162 |
|
| 163 |
-
Two cards are booked per request: this Space's denoise loop and the conditioner's forward
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
an `xlarge` booking costs **twice** its seconds, so the conditioner books the encode (45 s) and a prompt upsample
|
| 180 |
-
(60 s) as **two separate calls**, where one combined booking would be refused outright.
|
| 181 |
|
| 182 |
## Secrets
|
| 183 |
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
|
| 189 |
## Where diffusers comes from
|
| 190 |
|
|
|
|
| 160 |
|
| 161 |
## Whose GPU quota pays
|
| 162 |
|
| 163 |
+
Two cards are booked per request: this Space's denoise loop and the conditioner's forward, and **both are meant to be
|
| 164 |
+
billed to the requesting user**. ZeroGPU attributes a booking to the `X-IP-Token` of the request that triggered it —
|
| 165 |
+
its `/schedule` hands that token to the Spaces API together with the duration and the calling pod's IP, and nothing
|
| 166 |
+
else, so there is no Space identity in the decision and a valid token minted anywhere is honoured. This Space
|
| 167 |
+
therefore forwards the caller's header to the conditioner
|
| 168 |
+
(`gradio_client.Client(..., headers={"X-IP-Token": ...})`, off the `gr.Request` gradio injects on
|
| 169 |
+
the UI path and the `/generate` API path alike) rather than spending a token of its own.
|
| 170 |
+
|
| 171 |
+
A token ZeroGPU refuses — it answers `401`, which `spaces` surfaces as `Expired ZeroGPU proxy token` — falls back to
|
| 172 |
+
calling the conditioner with no token at all, billed to this Space's pod IP off a small shared quota. That is a safety
|
| 173 |
+
net rather than the intended path, and the log line `conditioner call paid for by ...` records which identity actually
|
| 174 |
+
paid, next to a decoded dump of what the incoming token claimed.
|
| 175 |
+
|
| 176 |
+
The conditioner is sized so that even the fallback is legal: an unattributed caller may book at most 120 credits at a
|
| 177 |
+
time and an `xlarge` booking costs **twice** its seconds (`_gpu_size_units`), so the conditioner books the encode
|
| 178 |
+
(45 s) and a prompt upsample (60 s) as **two separate calls**, where one combined booking would be refused outright.
|
|
|
|
|
|
|
| 179 |
|
| 180 |
## Secrets
|
| 181 |
|
| 182 |
+
None are required. Everything this Space downloads is public — the
|
| 183 |
+
[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) checkpoint and the
|
| 184 |
+
[`multimodalart/minimax-h3-aoti`](https://huggingface.co/multimodalart/minimax-h3-aoti) packages — and the conditioner
|
| 185 |
+
is a public Space called on the requesting user's own ZeroGPU token, never on an org token.
|
| 186 |
|
| 187 |
## Where diffusers comes from
|
| 188 |
|
app.py
CHANGED
|
@@ -24,9 +24,6 @@ ATTENTION = os.environ.get("H3_ATTENTION", "_native_cudnn").lower()
|
|
| 24 |
GPU_DURATION = int(os.environ.get("H3_GPU_DURATION", "900"))
|
| 25 |
GPU_SIZE = os.environ.get("H3_GPU_SIZE", "xlarge")
|
| 26 |
ON_SPACES = bool(os.environ.get("SPACE_ID"))
|
| 27 |
-
# Used for one thing only: paying for the conditioner call when the caller's own token cannot. Everything
|
| 28 |
-
# this Space downloads — the checkpoint and the AoTI packages — is public.
|
| 29 |
-
HF_TOKEN = os.environ.get("HF_TOKEN")
|
| 30 |
|
| 31 |
CANVASES = {
|
| 32 |
# 16:9
|
|
@@ -186,46 +183,37 @@ def conditioner(ip_token: str | None = None, hf_token: str | None = None):
|
|
| 186 |
key = (ip_token, hf_token)
|
| 187 |
if key in CLIENTS:
|
| 188 |
return CLIENTS[key]
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
headers={"X-IP-Token": ip_token} if ip_token else None,
|
| 193 |
-
)
|
| 194 |
if len(CLIENTS) >= 32:
|
| 195 |
CLIENTS.pop(next(iter(CLIENTS)))
|
| 196 |
CLIENTS[key] = client
|
| 197 |
return client
|
| 198 |
|
| 199 |
|
| 200 |
-
# What ZeroGPU says when an identity cannot pay for
|
| 201 |
-
#
|
| 202 |
-
# exhausted quota. All three mean "try the next identity" rather than "fail the request".
|
| 203 |
_UNPAYABLE = ("proxy token", "ZeroGPU quota", "larger than the maximum allowed", "GPU limit")
|
| 204 |
|
| 205 |
|
| 206 |
def call_conditioner(ip_token, **arguments):
|
| 207 |
-
"""One conditioner call,
|
| 208 |
-
|
| 209 |
-
In order of preference:
|
| 210 |
|
| 211 |
-
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
| 216 |
-
3. **no token at all** — an IP-based free quota, shared by everything calling out of this Space's egress IP and
|
| 217 |
-
worth a couple of requests a day. A last resort, not a design.
|
| 218 |
|
| 219 |
-
|
| 220 |
-
|
|
|
|
| 221 |
"""
|
| 222 |
api_name = arguments.pop("api_name")
|
| 223 |
-
attempts = []
|
| 224 |
if ip_token:
|
| 225 |
-
attempts.
|
| 226 |
-
if HF_TOKEN:
|
| 227 |
-
attempts.append(("this Space's HF_TOKEN", {"hf_token": HF_TOKEN}))
|
| 228 |
-
attempts.append(("no token, on an IP quota", {}))
|
| 229 |
|
| 230 |
for index, (label, identity) in enumerate(attempts):
|
| 231 |
try:
|
|
@@ -235,23 +223,39 @@ def call_conditioner(ip_token, **arguments):
|
|
| 235 |
except Exception as error:
|
| 236 |
if index == len(attempts) - 1 or not any(reason in str(error) for reason in _UNPAYABLE):
|
| 237 |
raise
|
| 238 |
-
print(f"[{LOG_TAG}] {label}: {error};
|
| 239 |
CLIENTS.pop((identity.get("ip_token"), identity.get("hf_token")), None)
|
| 240 |
|
| 241 |
|
| 242 |
def ip_token_of(request) -> str | None:
|
| 243 |
-
"""The
|
| 244 |
|
| 245 |
-
|
| 246 |
-
|
| 247 |
-
|
|
|
|
| 248 |
"""
|
| 249 |
headers = getattr(request, "headers", None)
|
| 250 |
token = None if headers is None else headers.get("x-ip-token")
|
| 251 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 252 |
return token
|
| 253 |
|
| 254 |
-
|
| 255 |
def encode_remote(prompt, image_path, last_image_path, canvas, num_frames, rewrite_prompt=False, ip_token=None):
|
| 256 |
"""Ask the conditioner Space for `prompt_embeds` + `text_token_tags`. Off this Space's GPU time entirely.
|
| 257 |
|
|
|
|
| 24 |
GPU_DURATION = int(os.environ.get("H3_GPU_DURATION", "900"))
|
| 25 |
GPU_SIZE = os.environ.get("H3_GPU_SIZE", "xlarge")
|
| 26 |
ON_SPACES = bool(os.environ.get("SPACE_ID"))
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
CANVASES = {
|
| 29 |
# 16:9
|
|
|
|
| 183 |
key = (ip_token, hf_token)
|
| 184 |
if key in CLIENTS:
|
| 185 |
return CLIENTS[key]
|
| 186 |
+
# No token of this Space's own: the booking is meant to be the caller's, and an unattributed call is the
|
| 187 |
+
# fallback rather than a second identity to spend.
|
| 188 |
+
client = Client(CONDITIONER_SPACE, headers={"X-IP-Token": ip_token} if ip_token else None)
|
|
|
|
|
|
|
| 189 |
if len(CLIENTS) >= 32:
|
| 190 |
CLIENTS.pop(next(iter(CLIENTS)))
|
| 191 |
CLIENTS[key] = client
|
| 192 |
return client
|
| 193 |
|
| 194 |
|
| 195 |
+
# What ZeroGPU says when an identity cannot pay for a booking: a proxy token its `/usage-approval` refused (`401`,
|
| 196 |
+
# surfaced as "Expired ZeroGPU proxy token"), a duration past what that identity may book, and an exhausted quota.
|
|
|
|
| 197 |
_UNPAYABLE = ("proxy token", "ZeroGPU quota", "larger than the maximum allowed", "GPU limit")
|
| 198 |
|
| 199 |
|
| 200 |
def call_conditioner(ip_token, **arguments):
|
| 201 |
+
"""One conditioner call, billed to **the requesting user**.
|
|
|
|
|
|
|
| 202 |
|
| 203 |
+
ZeroGPU attributes a booking to the `X-IP-Token` of the request that triggered it: `/schedule` hands that token to
|
| 204 |
+
the Spaces API's `/usage-approval` together with the duration and the calling pod's IP, and nothing else — there is
|
| 205 |
+
no Space identity in that call, so a *valid* token minted anywhere is honoured and the user's own quota pays for
|
| 206 |
+
both halves of their request. That is the whole reason this Space forwards the header instead of spending a token
|
| 207 |
+
of its own.
|
|
|
|
|
|
|
| 208 |
|
| 209 |
+
A token the Spaces API refuses (`401`) falls back to calling the conditioner with no token, which is billed to this
|
| 210 |
+
Space's pod IP off a small shared quota. That is a safety net, not the intended path: the log line
|
| 211 |
+
`conditioner call paid for by ...` records which one actually paid, so a Space that keeps falling back is visible.
|
| 212 |
"""
|
| 213 |
api_name = arguments.pop("api_name")
|
| 214 |
+
attempts = [("no token, on this Space's shared IP quota", {})]
|
| 215 |
if ip_token:
|
| 216 |
+
attempts.insert(0, ("the requesting user's own ZeroGPU token", {"ip_token": ip_token}))
|
|
|
|
|
|
|
|
|
|
| 217 |
|
| 218 |
for index, (label, identity) in enumerate(attempts):
|
| 219 |
try:
|
|
|
|
| 223 |
except Exception as error:
|
| 224 |
if index == len(attempts) - 1 or not any(reason in str(error) for reason in _UNPAYABLE):
|
| 225 |
raise
|
| 226 |
+
print(f"[{LOG_TAG}] {label} was refused: {error}; falling back", flush=True)
|
| 227 |
CLIENTS.pop((identity.get("ip_token"), identity.get("hf_token")), None)
|
| 228 |
|
| 229 |
|
| 230 |
def ip_token_of(request) -> str | None:
|
| 231 |
+
"""The requesting user's ZeroGPU identity, as the Spaces router put it on this request.
|
| 232 |
|
| 233 |
+
Logged, decoded, on every request — never the token itself, only what it claims. ZeroGPU refuses a token its
|
| 234 |
+
`/usage-approval` considers expired, and that refusal is indistinguishable from a missing one in the outcome, so
|
| 235 |
+
the claims are what tell the two apart when a request ends up on the fallback quota. Both the UI path and the
|
| 236 |
+
`/generate` API path reach this through the same `gr.Request` gradio injects for a parameter annotated with it.
|
| 237 |
"""
|
| 238 |
headers = getattr(request, "headers", None)
|
| 239 |
token = None if headers is None else headers.get("x-ip-token")
|
| 240 |
+
if token is None:
|
| 241 |
+
print(f"[{LOG_TAG}] no X-IP-Token on this request; the conditioner call cannot be billed to the caller", flush=True)
|
| 242 |
+
return None
|
| 243 |
+
try:
|
| 244 |
+
import base64
|
| 245 |
+
import json
|
| 246 |
+
import time
|
| 247 |
+
|
| 248 |
+
payload = json.loads(base64.urlsafe_b64decode(f"{token.split('.')[1]}=="))
|
| 249 |
+
left = payload.get("exp", 0) - time.time()
|
| 250 |
+
print(
|
| 251 |
+
f"[{LOG_TAG}] X-IP-Token present: {left:.0f}s to expiry, claims "
|
| 252 |
+
f"{ {k: v for k, v in payload.items() if k in ('exp', 'iat', 'sub', 'aud', 'error', 'user')} }",
|
| 253 |
+
flush=True,
|
| 254 |
+
)
|
| 255 |
+
except Exception as error: # a token that cannot be read is still worth forwarding; ZeroGPU is the judge
|
| 256 |
+
print(f"[{LOG_TAG}] X-IP-Token present but unreadable ({type(error).__name__}: {error})", flush=True)
|
| 257 |
return token
|
| 258 |
|
|
|
|
| 259 |
def encode_remote(prompt, image_path, last_image_path, canvas, num_frames, rewrite_prompt=False, ip_token=None):
|
| 260 |
"""Ask the conditioner Space for `prompt_embeds` + `text_token_tags`. Off this Space's GPU time entirely.
|
| 261 |
|