billingsmoore Claude Sonnet 5 commited on
Commit
9e83c68
·
1 Parent(s): 9eacf15

Enable ZeroGPU on local backend, remove deprecated Docker files

Browse files

HF requires at least one @spaces.GPU function to boot a gradio-sdk Space
on ZeroGPU hardware; decorate the local model's translate_batch and move
tensors to cuda when available. Drop the unused Dockerfile now that the
Space runs on the gradio SDK instead of docker.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Files changed (3) hide show
  1. Dockerfile +0 -14
  2. engine/local_backend.py +10 -1
  3. requirements.txt +1 -0
Dockerfile DELETED
@@ -1,14 +0,0 @@
1
- FROM python:3.11-slim-bookworm
2
-
3
- ENV PYTHONUNBUFFERED=1
4
-
5
- WORKDIR /app
6
-
7
- COPY requirements.txt .
8
- RUN pip install --no-cache-dir -r requirements.txt
9
-
10
- COPY . .
11
-
12
- EXPOSE 7860
13
-
14
- CMD ["python", "app.py"]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
engine/local_backend.py CHANGED
@@ -11,8 +11,14 @@ version that ships AutoModelForSeq2SeqLM/AutoTokenizer.
11
 
12
  This backend ignores the editable translation prompt — it's a plain
13
  seq2seq model, not an instruction-following LLM.
 
 
 
 
14
  """
15
 
 
 
16
  MODEL_ID = "billingsmoore/mlotsawa-ground-base"
17
  _PREFIX = "translate Tibetan to English: "
18
 
@@ -31,15 +37,18 @@ def _load():
31
  return _model, _tokenizer
32
 
33
 
 
34
  def translate_batch(texts: list[str]) -> list[str]:
35
  if not texts:
36
  return []
37
  import torch
38
  model, tokenizer = _load()
 
 
39
  inputs = tokenizer(
40
  [_PREFIX + t for t in texts],
41
  return_tensors="pt", padding=True, truncation=True,
42
- )
43
  with torch.no_grad():
44
  outputs = model.generate(**inputs, max_length=300, num_beams=4, early_stopping=True)
45
  return [tokenizer.decode(o, skip_special_tokens=True) for o in outputs]
 
11
 
12
  This backend ignores the editable translation prompt — it's a plain
13
  seq2seq model, not an instruction-following LLM.
14
+
15
+ translate_batch is decorated with @spaces.GPU so this runs on HF ZeroGPU
16
+ Spaces (which refuse to boot a gradio-sdk app with no @spaces.GPU function
17
+ at all); the decorator is a no-op outside a ZeroGPU Space.
18
  """
19
 
20
+ import spaces
21
+
22
  MODEL_ID = "billingsmoore/mlotsawa-ground-base"
23
  _PREFIX = "translate Tibetan to English: "
24
 
 
37
  return _model, _tokenizer
38
 
39
 
40
+ @spaces.GPU
41
  def translate_batch(texts: list[str]) -> list[str]:
42
  if not texts:
43
  return []
44
  import torch
45
  model, tokenizer = _load()
46
+ device = "cuda" if torch.cuda.is_available() else "cpu"
47
+ model.to(device)
48
  inputs = tokenizer(
49
  [_PREFIX + t for t in texts],
50
  return_tensors="pt", padding=True, truncation=True,
51
+ ).to(device)
52
  with torch.no_grad():
53
  outputs = model.generate(**inputs, max_length=300, num_beams=4, early_stopping=True)
54
  return [tokenizer.decode(o, skip_special_tokens=True) for o in outputs]
requirements.txt CHANGED
@@ -1,4 +1,5 @@
1
  gradio>=6.9.0
 
2
  google-genai
3
  python-dotenv
4
  transformers
 
1
  gradio>=6.9.0
2
+ spaces
3
  google-genai
4
  python-dotenv
5
  transformers