diyclassics Claude Opus 4.8 (1M context) commited on
Commit
56f42fb
·
1 Parent(s): b949170

fix: remove background model preload (degraded perf on cpu-basic)

Browse files

The daemon-thread preload competed for HF cpu-basic's 2 vCPUs and held a
load lock, so page reloads blocked behind it and felt very slow — the
opposite of the goal. Drop it entirely and reduce model_helpers to the
plain @st .cache_resource loader, which already guarantees a single
process-wide load reused across sessions/reruns with nothing that can
block a reload. Keeps the real win: the parser and custom-label demos no
longer reload the model on every interaction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files changed (2) hide show
  1. app.py +0 -7
  2. model_helpers.py +12 -45
app.py CHANGED
@@ -1,17 +1,10 @@
1
  import streamlit as st
2
 
3
- from model_helpers import start_preload
4
-
5
  st.set_page_config(
6
  page_title="LatinCy Dashboard | Home",
7
  page_icon="🏠",
8
  )
9
 
10
- # Warm the default lg model in a background thread while the visitor reads this
11
- # page, so the first demo they open finds it already loaded. Non-blocking and
12
- # idempotent — safe to call on every render.
13
- start_preload()
14
-
15
  st.write("# LatinCy Dashboard")
16
 
17
  st.sidebar.success("Select a demo above.")
 
1
  import streamlit as st
2
 
 
 
3
  st.set_page_config(
4
  page_title="LatinCy Dashboard | Home",
5
  page_icon="🏠",
6
  )
7
 
 
 
 
 
 
8
  st.write("# LatinCy Dashboard")
9
 
10
  st.sidebar.success("Select a demo above.")
model_helpers.py CHANGED
@@ -1,63 +1,30 @@
1
- """Shared spaCy model loading + background preload for the dashboard.
2
 
3
- Read-only demos import ``load_model()`` from here so they all share ONE model
4
  instance per name across the whole process, instead of each page caching its
5
  own copy (which multiplied a ~500 MB model across pages in RAM).
6
- ``start_preload()`` warms the default model in a background thread from the
7
- home page, so navigating into a demo finds the model already resident.
8
 
9
  Demos that MUTATE their pipeline (e.g. the custom-label demo, which adds a
10
- component with ``add_pipe``) must NOT use this shared instance — they should
11
- cache their own copy — because mutating the shared model would leak the change
12
- into every other demo.
13
  """
14
 
15
- import threading
16
-
17
  import spacy
18
  import streamlit as st
19
 
20
  DEFAULT_MODEL = "la_core_web_lg"
21
 
22
- # Single source of truth for loaded models, shared between the background
23
- # preloader and the synchronous loader. Guarded by _lock so a preload and a
24
- # concurrent load_model() can never start two loads of the same model.
25
- _models = {}
26
- _lock = threading.Lock()
27
-
28
-
29
- def _load(model_name):
30
- """Load a model with no Streamlit (st.*) calls — safe off the main thread."""
31
- nlp = spacy.load(model_name)
32
- if "trf_vectors" in nlp.pipe_names: # heavy, unused in these demos
33
- nlp.disable_pipe("trf_vectors")
34
- return nlp
35
-
36
-
37
- def _get(model_name):
38
- with _lock:
39
- if model_name not in _models:
40
- _models[model_name] = _load(model_name)
41
- return _models[model_name]
42
-
43
-
44
- def start_preload(model_name=DEFAULT_MODEL):
45
- """Kick off a non-blocking background load of the default model.
46
-
47
- Called from the home page so the model warms while the user reads the demo
48
- list. Cheap to call repeatedly: once loaded, the worker returns at once.
49
- """
50
- if model_name in _models:
51
- return
52
- threading.Thread(target=_get, args=(model_name,), daemon=True).start()
53
-
54
 
55
  @st.cache_resource(show_spinner="Loading LatinCy model…")
56
  def load_model(model_name=DEFAULT_MODEL):
57
  """Return the shared, cached spaCy model for ``model_name``.
58
 
59
- Reuses the background-preloaded instance when ready; otherwise loads it once
60
- synchronously. Read-only use only do not ``add_pipe``/``disable`` on the
61
- result, since every demo shares this instance.
62
  """
63
- return _get(model_name)
 
 
 
 
1
+ """Shared spaCy model loading for the dashboard.
2
 
3
+ Read-only demos import ``load_model()`` so they all share ONE cached model
4
  instance per name across the whole process, instead of each page caching its
5
  own copy (which multiplied a ~500 MB model across pages in RAM).
6
+ ``@st.cache_resource`` already guarantees the model loads once per process and
7
+ is reused across sessions/reruns no manual threading needed.
8
 
9
  Demos that MUTATE their pipeline (e.g. the custom-label demo, which adds a
10
+ component with ``add_pipe``) must cache their OWN instance instead mutating
11
+ this shared one would leak the change into every other demo.
 
12
  """
13
 
 
 
14
  import spacy
15
  import streamlit as st
16
 
17
  DEFAULT_MODEL = "la_core_web_lg"
18
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
  @st.cache_resource(show_spinner="Loading LatinCy model…")
21
  def load_model(model_name=DEFAULT_MODEL):
22
  """Return the shared, cached spaCy model for ``model_name``.
23
 
24
+ Read-only use only do not ``add_pipe``/``disable`` on the result, since
25
+ every read-only demo shares this one instance.
 
26
  """
27
+ nlp = spacy.load(model_name)
28
+ if "trf_vectors" in nlp.pipe_names: # heavy, unused in these demos
29
+ nlp.disable_pipe("trf_vectors")
30
+ return nlp