Instructions to use flyingfishinwater/good_and_small_models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use flyingfishinwater/good_and_small_models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf flyingfishinwater/good_and_small_models:Q4_K_M # Run inference directly in the terminal: llama cli -hf flyingfishinwater/good_and_small_models:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf flyingfishinwater/good_and_small_models:Q4_K_M # Run inference directly in the terminal: llama cli -hf flyingfishinwater/good_and_small_models:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf flyingfishinwater/good_and_small_models:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf flyingfishinwater/good_and_small_models:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf flyingfishinwater/good_and_small_models:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf flyingfishinwater/good_and_small_models:Q4_K_M
Use Docker
docker model run hf.co/flyingfishinwater/good_and_small_models:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use flyingfishinwater/good_and_small_models with Ollama:
ollama run hf.co/flyingfishinwater/good_and_small_models:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use flyingfishinwater/good_and_small_models with Docker Model Runner:
docker model run hf.co/flyingfishinwater/good_and_small_models:Q4_K_M
- Lemonade
How to use flyingfishinwater/good_and_small_models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull flyingfishinwater/good_and_small_models:Q4_K_M
Run and chat with the model
lemonade run user.good_and_small_models-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Update README.md
Browse files
README.md
CHANGED
|
@@ -18,11 +18,11 @@ It's a very small LLAMA2 model with only 460M parameters trained with 1T tokens.
|
|
| 18 |
|
| 19 |
**Context Length:** 1024 tokens
|
| 20 |
|
| 21 |
-
**Prompt Format:**
|
| 22 |
|
| 23 |
```
|
| 24 |
<human>: {{prompt}}
|
| 25 |
-
<bot>:
|
| 26 |
```
|
| 27 |
|
| 28 |
**Template Name:** TinyLlama
|
|
@@ -56,10 +56,10 @@ The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens.
|
|
| 56 |
|
| 57 |
**Context Length:** 4096 tokens
|
| 58 |
|
| 59 |
-
**Prompt Format:**
|
| 60 |
|
| 61 |
```
|
| 62 |
-
<|system|>You are a friendly chatbot who always responds in the style of a pirate.</s><|user|>{{prompt}}</s><|assistant|>
|
| 63 |
```
|
| 64 |
|
| 65 |
**Template Name:** TinyLlama
|
|
@@ -93,10 +93,10 @@ The Mistral-7B-v0.2 Large Language Model (LLM) is a pretrained generative text m
|
|
| 93 |
|
| 94 |
**Context Length:** 4096 tokens
|
| 95 |
|
| 96 |
-
**Prompt Format:**
|
| 97 |
|
| 98 |
```
|
| 99 |
-
<s>[INST]{{prompt}}[/INST]</s>
|
| 100 |
```
|
| 101 |
|
| 102 |
**Template Name:** Mistral
|
|
@@ -130,10 +130,10 @@ OpenChat is an innovative library of open-source language models, fine-tuned wit
|
|
| 130 |
|
| 131 |
**Context Length:** 4096 tokens
|
| 132 |
|
| 133 |
-
**Prompt Format:**
|
| 134 |
|
| 135 |
```
|
| 136 |
-
<s>[INST]{{prompt}}[/INST]</s>
|
| 137 |
```
|
| 138 |
|
| 139 |
**Template Name:** Mistral
|
|
@@ -167,11 +167,11 @@ Phi-2 is a Transformer with 2.7 billion parameters. It was trained using the sam
|
|
| 167 |
|
| 168 |
**Context Length:** 4096 tokens
|
| 169 |
|
| 170 |
-
**Prompt Format:**
|
| 171 |
|
| 172 |
```
|
| 173 |
Instruct: {{prompt}}
|
| 174 |
-
Output:
|
| 175 |
```
|
| 176 |
|
| 177 |
**Template Name:** PHI
|
|
@@ -205,14 +205,14 @@ The Yi series models are the next generation of open-source large language model
|
|
| 205 |
|
| 206 |
**Context Length:** 200000 tokens
|
| 207 |
|
| 208 |
-
**Prompt Format:**
|
| 209 |
|
| 210 |
```
|
| 211 |
<|im_start|>user
|
| 212 |
<|im_end|>
|
| 213 |
{{prompt}}
|
| 214 |
<|im_start|>assistant
|
| 215 |
-
|
| 216 |
```
|
| 217 |
|
| 218 |
**Template Name:** yi
|
|
@@ -246,13 +246,13 @@ Gemma is a family of lightweight, state-of-the-art open models built from the sa
|
|
| 246 |
|
| 247 |
**Context Length:** 8192 tokens
|
| 248 |
|
| 249 |
-
**Prompt Format:**
|
| 250 |
|
| 251 |
```
|
| 252 |
<bos><start_of_turn>user
|
| 253 |
{{prompt}}<end_of_turn>
|
| 254 |
<start_of_turn>model
|
| 255 |
-
|
| 256 |
```
|
| 257 |
|
| 258 |
**Template Name:** gemma
|
|
@@ -286,12 +286,12 @@ StarCoder2-3B model is a 3B parameter model trained on 17 programming languages
|
|
| 286 |
|
| 287 |
**Context Length:** 8192 tokens
|
| 288 |
|
| 289 |
-
**Prompt Format:**
|
| 290 |
|
| 291 |
```
|
| 292 |
### Instruction
|
| 293 |
{{prompt}}### Response
|
| 294 |
-
|
| 295 |
```
|
| 296 |
|
| 297 |
**Template Name:** starcoder
|
|
@@ -325,14 +325,14 @@ Chinese Tiny LLM 2B 是首个以中文为中心的大型语言模型,主要在
|
|
| 325 |
|
| 326 |
**Context Length:** 4096 tokens
|
| 327 |
|
| 328 |
-
**Prompt Format:**
|
| 329 |
|
| 330 |
```
|
| 331 |
<|im_start|>user
|
| 332 |
{{prompt}}
|
| 333 |
<|im_end|>
|
| 334 |
<|im_start|>assistant
|
| 335 |
-
|
| 336 |
```
|
| 337 |
|
| 338 |
**Template Name:** chatml
|
|
@@ -366,14 +366,14 @@ This model is based on Mistral-7b-v0.2 with 16k context lengths. It's a uncensor
|
|
| 366 |
|
| 367 |
**Context Length:** 16384 tokens
|
| 368 |
|
| 369 |
-
**Prompt Format:**
|
| 370 |
|
| 371 |
```
|
| 372 |
<|im_start|>user
|
| 373 |
{{prompt}}
|
| 374 |
<|im_end|>
|
| 375 |
<|im_start|>assistant
|
| 376 |
-
|
| 377 |
```
|
| 378 |
|
| 379 |
**Template Name:** chatml
|
|
|
|
| 18 |
|
| 19 |
**Context Length:** 1024 tokens
|
| 20 |
|
| 21 |
+
**Prompt Format:**
|
| 22 |
|
| 23 |
```
|
| 24 |
<human>: {{prompt}}
|
| 25 |
+
<bot>:
|
| 26 |
```
|
| 27 |
|
| 28 |
**Template Name:** TinyLlama
|
|
|
|
| 56 |
|
| 57 |
**Context Length:** 4096 tokens
|
| 58 |
|
| 59 |
+
**Prompt Format:**
|
| 60 |
|
| 61 |
```
|
| 62 |
+
<|system|>You are a friendly chatbot who always responds in the style of a pirate.</s><|user|>{{prompt}}</s><|assistant|>
|
| 63 |
```
|
| 64 |
|
| 65 |
**Template Name:** TinyLlama
|
|
|
|
| 93 |
|
| 94 |
**Context Length:** 4096 tokens
|
| 95 |
|
| 96 |
+
**Prompt Format:**
|
| 97 |
|
| 98 |
```
|
| 99 |
+
<s>[INST]{{prompt}}[/INST]</s>
|
| 100 |
```
|
| 101 |
|
| 102 |
**Template Name:** Mistral
|
|
|
|
| 130 |
|
| 131 |
**Context Length:** 4096 tokens
|
| 132 |
|
| 133 |
+
**Prompt Format:**
|
| 134 |
|
| 135 |
```
|
| 136 |
+
<s>[INST]{{prompt}}[/INST]</s>
|
| 137 |
```
|
| 138 |
|
| 139 |
**Template Name:** Mistral
|
|
|
|
| 167 |
|
| 168 |
**Context Length:** 4096 tokens
|
| 169 |
|
| 170 |
+
**Prompt Format:**
|
| 171 |
|
| 172 |
```
|
| 173 |
Instruct: {{prompt}}
|
| 174 |
+
Output:
|
| 175 |
```
|
| 176 |
|
| 177 |
**Template Name:** PHI
|
|
|
|
| 205 |
|
| 206 |
**Context Length:** 200000 tokens
|
| 207 |
|
| 208 |
+
**Prompt Format:**
|
| 209 |
|
| 210 |
```
|
| 211 |
<|im_start|>user
|
| 212 |
<|im_end|>
|
| 213 |
{{prompt}}
|
| 214 |
<|im_start|>assistant
|
| 215 |
+
|
| 216 |
```
|
| 217 |
|
| 218 |
**Template Name:** yi
|
|
|
|
| 246 |
|
| 247 |
**Context Length:** 8192 tokens
|
| 248 |
|
| 249 |
+
**Prompt Format:**
|
| 250 |
|
| 251 |
```
|
| 252 |
<bos><start_of_turn>user
|
| 253 |
{{prompt}}<end_of_turn>
|
| 254 |
<start_of_turn>model
|
| 255 |
+
|
| 256 |
```
|
| 257 |
|
| 258 |
**Template Name:** gemma
|
|
|
|
| 286 |
|
| 287 |
**Context Length:** 8192 tokens
|
| 288 |
|
| 289 |
+
**Prompt Format:**
|
| 290 |
|
| 291 |
```
|
| 292 |
### Instruction
|
| 293 |
{{prompt}}### Response
|
| 294 |
+
|
| 295 |
```
|
| 296 |
|
| 297 |
**Template Name:** starcoder
|
|
|
|
| 325 |
|
| 326 |
**Context Length:** 4096 tokens
|
| 327 |
|
| 328 |
+
**Prompt Format:**
|
| 329 |
|
| 330 |
```
|
| 331 |
<|im_start|>user
|
| 332 |
{{prompt}}
|
| 333 |
<|im_end|>
|
| 334 |
<|im_start|>assistant
|
| 335 |
+
|
| 336 |
```
|
| 337 |
|
| 338 |
**Template Name:** chatml
|
|
|
|
| 366 |
|
| 367 |
**Context Length:** 16384 tokens
|
| 368 |
|
| 369 |
+
**Prompt Format:**
|
| 370 |
|
| 371 |
```
|
| 372 |
<|im_start|>user
|
| 373 |
{{prompt}}
|
| 374 |
<|im_end|>
|
| 375 |
<|im_start|>assistant
|
| 376 |
+
|
| 377 |
```
|
| 378 |
|
| 379 |
**Template Name:** chatml
|