Instructions to use flyingfishinwater/good_and_small_models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use flyingfishinwater/good_and_small_models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf flyingfishinwater/good_and_small_models:Q4_K_M # Run inference directly in the terminal: llama cli -hf flyingfishinwater/good_and_small_models:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf flyingfishinwater/good_and_small_models:Q4_K_M # Run inference directly in the terminal: llama cli -hf flyingfishinwater/good_and_small_models:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf flyingfishinwater/good_and_small_models:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf flyingfishinwater/good_and_small_models:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf flyingfishinwater/good_and_small_models:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf flyingfishinwater/good_and_small_models:Q4_K_M
Use Docker
docker model run hf.co/flyingfishinwater/good_and_small_models:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use flyingfishinwater/good_and_small_models with Ollama:
ollama run hf.co/flyingfishinwater/good_and_small_models:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use flyingfishinwater/good_and_small_models with Docker Model Runner:
docker model run hf.co/flyingfishinwater/good_and_small_models:Q4_K_M
- Lemonade
How to use flyingfishinwater/good_and_small_models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull flyingfishinwater/good_and_small_models:Q4_K_M
Run and chat with the model
lemonade run user.good_and_small_models-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Update README.md
Browse files
README.md
CHANGED
|
@@ -385,4 +385,43 @@ This model is based on Mistral-7b-v0.2 with 16k context lengths. It's a uncensor
|
|
| 385 |
|
| 386 |
**Add EOS Token:** No
|
| 387 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 388 |
**Parse Special Tokens:** Yes
|
|
|
|
| 385 |
|
| 386 |
**Add EOS Token:** No
|
| 387 |
|
| 388 |
+
**Parse Special Tokens:** Yes
|
| 389 |
+
|
| 390 |
+
|
| 391 |
+
---
|
| 392 |
+
|
| 393 |
+
# WizardLM-2 7B
|
| 394 |
+
|
| 395 |
+
The WizardLM-2 is one of the next generation state-of-the-art large language models, which have improved performance on complex chat, multilingual, reasoning and agent.
|
| 396 |
+
|
| 397 |
+
**Model Intention:** It's a state-of-the-art large language model with improved performance on complex chat, multilingual, reasoning and agent.
|
| 398 |
+
|
| 399 |
+
**Model URL:** [https://huggingface.co/flyingfishinwater/goodmodels/resolve/main/WizardLM-2-7B.Q3_K_M.gguf?download=true](https://huggingface.co/flyingfishinwater/goodmodels/resolve/main/WizardLM-2-7B.Q3_K_M.gguf?download=true)
|
| 400 |
+
|
| 401 |
+
**Model Info URL:** [https://huggingface.co/MaziyarPanahi/WizardLM-2-7B-GGUF](https://huggingface.co/MaziyarPanahi/WizardLM-2-7B-GGUF)
|
| 402 |
+
|
| 403 |
+
**Model License:** [License Info](https://www.apache.org/licenses/LICENSE-2.0)
|
| 404 |
+
|
| 405 |
+
**Model Description:** The WizardLM-2 is one of the next generation state-of-the-art large language models, which have improved performance on complex chat, multilingual, reasoning and agent.
|
| 406 |
+
|
| 407 |
+
**Developer:** [https://huggingface.co/collections/microsoft/wizardlm-661d403f71e6c8257dbd598a](https://huggingface.co/collections/microsoft/wizardlm-661d403f71e6c8257dbd598a)
|
| 408 |
+
|
| 409 |
+
**File Size:** 3519 MB
|
| 410 |
+
|
| 411 |
+
**Context Length:** 32768 tokens
|
| 412 |
+
|
| 413 |
+
**Prompt Format:**
|
| 414 |
+
|
| 415 |
+
```
|
| 416 |
+
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
|
| 417 |
+
USER: {{prompt}}
|
| 418 |
+
ASSISTANT:
|
| 419 |
+
```
|
| 420 |
+
|
| 421 |
+
**Template Name:** chatml
|
| 422 |
+
|
| 423 |
+
**Add BOS Token:** Yes
|
| 424 |
+
|
| 425 |
+
**Add EOS Token:** No
|
| 426 |
+
|
| 427 |
**Parse Special Tokens:** Yes
|