Chia-Chi commited on
Commit
2517022
Β·
verified Β·
1 Parent(s): 2c073a5

Add model card

Browse files
Files changed (1) hide show
  1. README.md +64 -0
README.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - paligemma2
6
+ - lora
7
+ - multi-lora
8
+ - serving
9
+ - openai-api
10
+ - visual-inspection
11
+ - peft
12
+ - google
13
+ license: apache-2.0
14
+ ---
15
+
16
+ # Anchor β€” PaliGemma2 Multi-LoRA Server
17
+
18
+ **Load multiple LoRA adapters once. Switch between them at inference time β€” 216ms, no reload.**
19
+
20
+ β†’ **GitHub:** [recursia-lab/anchor](https://github.com/recursia-lab/anchor)
21
+
22
+ ## What is this?
23
+
24
+ Anchor is a lightweight serving server for PaliGemma2 with multiple LoRA adapters.
25
+ Unlike frameworks that load adapters per-request from disk, Anchor keeps all adapters
26
+ in GPU memory simultaneously β€” switching is a pointer swap.
27
+
28
+ ```
29
+ Request: model="open_circuit" β†’ set_adapter() β†’ generate() β†’ 216ms
30
+ Request: model="missing_hole" β†’ set_adapter() β†’ generate() β†’ 216ms
31
+ Request: model="base" β†’ disable_adapters() β†’ generate()
32
+ ```
33
+
34
+ ## Quick Start
35
+
36
+ ```bash
37
+ git clone https://github.com/recursia-lab/anchor
38
+ docker build -t anchor .
39
+ docker run --gpus all -v /model:/model -v /lora:/lora -p 8080:8080 anchor
40
+ ```
41
+
42
+ ## API (OpenAI-compatible)
43
+
44
+ ```bash
45
+ curl http://localhost:8080/v1/chat/completions \
46
+ -d '{"model": "your_adapter", "messages": [...]}'
47
+ ```
48
+
49
+ ## Framework Support
50
+
51
+ | Framework | PaliGemma2 LoRA |
52
+ |---|---|
53
+ | **Anchor** | βœ… pre-loaded, 216ms switch |
54
+ | vLLM | βœ… per-request load |
55
+ | SGLang | 🚧 [PR #24034](https://github.com/sgl-project/sglang/pull/24034) |
56
+
57
+ ## Community Adapters
58
+
59
+ See [recursia-lab/paligemma2-adapters](https://github.com/recursia-lab/paligemma2-adapters)
60
+ for a curated index of community fine-tuned PaliGemma2 LoRA adapters.
61
+
62
+ ---
63
+
64
+ Built by [Recursia Lab](https://github.com/recursia-lab) β€’ Apache 2.0