multimodalart HF Staff commited on
Commit
9a812e9
·
verified ·
1 Parent(s): 8e346cd

Bill the conditioner call to the requesting user's own ZeroGPU token, never an org token

Browse files
Files changed (2) hide show
  1. README.md +20 -22
  2. app.py +39 -35
README.md CHANGED
@@ -160,31 +160,29 @@ one-time `PIPE.to("cuda")` is inside the first row's 339 s and does not reappear
160
 
161
  ## Whose GPU quota pays
162
 
163
- Two cards are booked per request: this Space's denoise loop and the conditioner's forward. ZeroGPU attributes a
164
- booking to the identity of the request that triggered it, so the conditioner call tries three in order:
165
-
166
- 1. **the caller's own `X-IP-Token`**, forwarded off the `gr.Request` gradio injects (the UI path and the `/generate`
167
- API path alike). The request then bills as one request across both halves and costs this org nothing. Best effort:
168
- ZeroGPU answers `401` for a proxy token it will not honour — which is what a token minted for *this* Space looks
169
- like arriving at another one — and `spaces` surfaces that as `Expired ZeroGPU proxy token`.
170
- 2. **this Space's `HF_TOKEN`**, which charges the account that owns the Space and has a real quota. This is what
171
- carries the Space in practice.
172
- 3. **no token**, an IP-based free quota shared by everything calling out of this Space's egress IP. A last resort.
173
-
174
- Any of ZeroGPU's "this identity cannot pay" answers — a refused proxy token, a duration past what the identity may
175
- book, an exhausted quota — moves on to the next identity instead of failing the request, and the log line
176
- `conditioner call paid for by ...` records which one paid.
177
-
178
- The conditioner is sized so that even (3) is legal: an unattributed caller may book at most 120 credits at a time and
179
- an `xlarge` booking costs **twice** its seconds, so the conditioner books the encode (45 s) and a prompt upsample
180
- (60 s) as **two separate calls**, where one combined booking would be refused outright.
181
 
182
  ## Secrets
183
 
184
- `HF_TOKEN` used for exactly one thing: paying for the conditioner call when the caller's own ZeroGPU token cannot
185
- (see above). Everything this Space downloads is public: the [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3)
186
- checkpoint and the [`multimodalart/minimax-h3-aoti`](https://huggingface.co/multimodalart/minimax-h3-aoti) packages.
187
- Without it, a call the caller cannot pay for falls back to a shared IP quota worth a couple of requests a day.
188
 
189
  ## Where diffusers comes from
190
 
 
160
 
161
  ## Whose GPU quota pays
162
 
163
+ Two cards are booked per request: this Space's denoise loop and the conditioner's forward, and **both are meant to be
164
+ billed to the requesting user**. ZeroGPU attributes a booking to the `X-IP-Token` of the request that triggered it —
165
+ its `/schedule` hands that token to the Spaces API together with the duration and the calling pod's IP, and nothing
166
+ else, so there is no Space identity in the decision and a valid token minted anywhere is honoured. This Space
167
+ therefore forwards the caller's header to the conditioner
168
+ (`gradio_client.Client(..., headers={"X-IP-Token": ...})`, off the `gr.Request` gradio injects on
169
+ the UI path and the `/generate` API path alike) rather than spending a token of its own.
170
+
171
+ A token ZeroGPU refuses — it answers `401`, which `spaces` surfaces as `Expired ZeroGPU proxy token` — falls back to
172
+ calling the conditioner with no token at all, billed to this Space's pod IP off a small shared quota. That is a safety
173
+ net rather than the intended path, and the log line `conditioner call paid for by ...` records which identity actually
174
+ paid, next to a decoded dump of what the incoming token claimed.
175
+
176
+ The conditioner is sized so that even the fallback is legal: an unattributed caller may book at most 120 credits at a
177
+ time and an `xlarge` booking costs **twice** its seconds (`_gpu_size_units`), so the conditioner books the encode
178
+ (45 s) and a prompt upsample (60 s) as **two separate calls**, where one combined booking would be refused outright.
 
 
179
 
180
  ## Secrets
181
 
182
+ None are required. Everything this Space downloads is public the
183
+ [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) checkpoint and the
184
+ [`multimodalart/minimax-h3-aoti`](https://huggingface.co/multimodalart/minimax-h3-aoti) packages — and the conditioner
185
+ is a public Space called on the requesting user's own ZeroGPU token, never on an org token.
186
 
187
  ## Where diffusers comes from
188
 
app.py CHANGED
@@ -24,9 +24,6 @@ ATTENTION = os.environ.get("H3_ATTENTION", "_native_cudnn").lower()
24
  GPU_DURATION = int(os.environ.get("H3_GPU_DURATION", "900"))
25
  GPU_SIZE = os.environ.get("H3_GPU_SIZE", "xlarge")
26
  ON_SPACES = bool(os.environ.get("SPACE_ID"))
27
- # Used for one thing only: paying for the conditioner call when the caller's own token cannot. Everything
28
- # this Space downloads — the checkpoint and the AoTI packages — is public.
29
- HF_TOKEN = os.environ.get("HF_TOKEN")
30
 
31
  CANVASES = {
32
  # 16:9
@@ -186,46 +183,37 @@ def conditioner(ip_token: str | None = None, hf_token: str | None = None):
186
  key = (ip_token, hf_token)
187
  if key in CLIENTS:
188
  return CLIENTS[key]
189
- client = Client(
190
- CONDITIONER_SPACE,
191
- token=hf_token,
192
- headers={"X-IP-Token": ip_token} if ip_token else None,
193
- )
194
  if len(CLIENTS) >= 32:
195
  CLIENTS.pop(next(iter(CLIENTS)))
196
  CLIENTS[key] = client
197
  return client
198
 
199
 
200
- # What ZeroGPU says when an identity cannot pay for the booking, in any of its forms: a proxy token it will not
201
- # honour (`401`, surfaced as "Expired ZeroGPU proxy token"), a duration past what that identity may book, and an
202
- # exhausted quota. All three mean "try the next identity" rather than "fail the request".
203
  _UNPAYABLE = ("proxy token", "ZeroGPU quota", "larger than the maximum allowed", "GPU limit")
204
 
205
 
206
  def call_conditioner(ip_token, **arguments):
207
- """One conditioner call, against the first ZeroGPU identity that can pay for it.
208
-
209
- In order of preference:
210
 
211
- 1. **the caller's own forwarded `X-IP-Token`** the request bills as one request across both halves and costs
212
- this org nothing. Best effort: a proxy token minted for this Space is not necessarily honoured when it
213
- arrives at another one, and ZeroGPU answers `401` when it is not.
214
- 2. **this Space's `HF_TOKEN`** the booking is charged to the account that owns the Space, which has a real
215
- quota. This is what carries the Space in practice.
216
- 3. **no token at all** — an IP-based free quota, shared by everything calling out of this Space's egress IP and
217
- worth a couple of requests a day. A last resort, not a design.
218
 
219
- The conditioner's own bookings are sized to fit even (3): it books an encode and a prompt upsample as two calls
220
- of 45 s and 60 s, because an unattributed caller may book at most 60 s of `xlarge` at a time.
 
221
  """
222
  api_name = arguments.pop("api_name")
223
- attempts = []
224
  if ip_token:
225
- attempts.append(("the caller's forwarded ZeroGPU token", {"ip_token": ip_token}))
226
- if HF_TOKEN:
227
- attempts.append(("this Space's HF_TOKEN", {"hf_token": HF_TOKEN}))
228
- attempts.append(("no token, on an IP quota", {}))
229
 
230
  for index, (label, identity) in enumerate(attempts):
231
  try:
@@ -235,23 +223,39 @@ def call_conditioner(ip_token, **arguments):
235
  except Exception as error:
236
  if index == len(attempts) - 1 or not any(reason in str(error) for reason in _UNPAYABLE):
237
  raise
238
- print(f"[{LOG_TAG}] {label}: {error}; trying the next identity", flush=True)
239
  CLIENTS.pop((identity.get("ip_token"), identity.get("hf_token")), None)
240
 
241
 
242
  def ip_token_of(request) -> str | None:
243
- """The caller's ZeroGPU identity, as the Spaces router put it on this request.
244
 
245
- Present on a browser request and on an API request the router could attribute; absent for a truly anonymous
246
- caller, which then falls back to the conditioner's IP-based quota. Both the UI path and the `/generate` API path
247
- reach this through the same `gr.Request` gradio injects for a parameter annotated with it.
 
248
  """
249
  headers = getattr(request, "headers", None)
250
  token = None if headers is None else headers.get("x-ip-token")
251
- print(f"[gen] conditioner call {'forwards the caller ZeroGPU token' if token else 'is anonymous (IP quota)'}", flush=True)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
252
  return token
253
 
254
-
255
  def encode_remote(prompt, image_path, last_image_path, canvas, num_frames, rewrite_prompt=False, ip_token=None):
256
  """Ask the conditioner Space for `prompt_embeds` + `text_token_tags`. Off this Space's GPU time entirely.
257
 
 
24
  GPU_DURATION = int(os.environ.get("H3_GPU_DURATION", "900"))
25
  GPU_SIZE = os.environ.get("H3_GPU_SIZE", "xlarge")
26
  ON_SPACES = bool(os.environ.get("SPACE_ID"))
 
 
 
27
 
28
  CANVASES = {
29
  # 16:9
 
183
  key = (ip_token, hf_token)
184
  if key in CLIENTS:
185
  return CLIENTS[key]
186
+ # No token of this Space's own: the booking is meant to be the caller's, and an unattributed call is the
187
+ # fallback rather than a second identity to spend.
188
+ client = Client(CONDITIONER_SPACE, headers={"X-IP-Token": ip_token} if ip_token else None)
 
 
189
  if len(CLIENTS) >= 32:
190
  CLIENTS.pop(next(iter(CLIENTS)))
191
  CLIENTS[key] = client
192
  return client
193
 
194
 
195
+ # What ZeroGPU says when an identity cannot pay for a booking: a proxy token its `/usage-approval` refused (`401`,
196
+ # surfaced as "Expired ZeroGPU proxy token"), a duration past what that identity may book, and an exhausted quota.
 
197
  _UNPAYABLE = ("proxy token", "ZeroGPU quota", "larger than the maximum allowed", "GPU limit")
198
 
199
 
200
  def call_conditioner(ip_token, **arguments):
201
+ """One conditioner call, billed to **the requesting user**.
 
 
202
 
203
+ ZeroGPU attributes a booking to the `X-IP-Token` of the request that triggered it: `/schedule` hands that token to
204
+ the Spaces API's `/usage-approval` together with the duration and the calling pod's IP, and nothing else there is
205
+ no Space identity in that call, so a *valid* token minted anywhere is honoured and the user's own quota pays for
206
+ both halves of their request. That is the whole reason this Space forwards the header instead of spending a token
207
+ of its own.
 
 
208
 
209
+ A token the Spaces API refuses (`401`) falls back to calling the conditioner with no token, which is billed to this
210
+ Space's pod IP off a small shared quota. That is a safety net, not the intended path: the log line
211
+ `conditioner call paid for by ...` records which one actually paid, so a Space that keeps falling back is visible.
212
  """
213
  api_name = arguments.pop("api_name")
214
+ attempts = [("no token, on this Space's shared IP quota", {})]
215
  if ip_token:
216
+ attempts.insert(0, ("the requesting user's own ZeroGPU token", {"ip_token": ip_token}))
 
 
 
217
 
218
  for index, (label, identity) in enumerate(attempts):
219
  try:
 
223
  except Exception as error:
224
  if index == len(attempts) - 1 or not any(reason in str(error) for reason in _UNPAYABLE):
225
  raise
226
+ print(f"[{LOG_TAG}] {label} was refused: {error}; falling back", flush=True)
227
  CLIENTS.pop((identity.get("ip_token"), identity.get("hf_token")), None)
228
 
229
 
230
  def ip_token_of(request) -> str | None:
231
+ """The requesting user's ZeroGPU identity, as the Spaces router put it on this request.
232
 
233
+ Logged, decoded, on every request never the token itself, only what it claims. ZeroGPU refuses a token its
234
+ `/usage-approval` considers expired, and that refusal is indistinguishable from a missing one in the outcome, so
235
+ the claims are what tell the two apart when a request ends up on the fallback quota. Both the UI path and the
236
+ `/generate` API path reach this through the same `gr.Request` gradio injects for a parameter annotated with it.
237
  """
238
  headers = getattr(request, "headers", None)
239
  token = None if headers is None else headers.get("x-ip-token")
240
+ if token is None:
241
+ print(f"[{LOG_TAG}] no X-IP-Token on this request; the conditioner call cannot be billed to the caller", flush=True)
242
+ return None
243
+ try:
244
+ import base64
245
+ import json
246
+ import time
247
+
248
+ payload = json.loads(base64.urlsafe_b64decode(f"{token.split('.')[1]}=="))
249
+ left = payload.get("exp", 0) - time.time()
250
+ print(
251
+ f"[{LOG_TAG}] X-IP-Token present: {left:.0f}s to expiry, claims "
252
+ f"{ {k: v for k, v in payload.items() if k in ('exp', 'iat', 'sub', 'aud', 'error', 'user')} }",
253
+ flush=True,
254
+ )
255
+ except Exception as error: # a token that cannot be read is still worth forwarding; ZeroGPU is the judge
256
+ print(f"[{LOG_TAG}] X-IP-Token present but unreadable ({type(error).__name__}: {error})", flush=True)
257
  return token
258
 
 
259
  def encode_remote(prompt, image_path, last_image_path, canvas, num_frames, rewrite_prompt=False, ip_token=None):
260
  """Ask the conditioner Space for `prompt_embeds` + `text_token_tags`. Off this Space's GPU time entirely.
261