Instructions to use mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF", dtype="auto") - llama-cpp-python
How to use mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF", filename="NinjaMouse-3B-40L-danube.i1-IQ1_M.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M
Use Docker
docker model run hf.co/mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF with Ollama:
ollama run hf.co/mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M
- Unsloth Studio
How to use mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF to start chatting
- Atomic Chat new
- Docker Model Runner
How to use mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF with Docker Model Runner:
docker model run hf.co/mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M
- Lemonade
How to use mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull mradermacher/NinjaMouse-3B-40L-danube-i1-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.NinjaMouse-3B-40L-danube-i1-GGUF-Q4_K_M
List all available models
lemonade list
uploaded from nico1
Browse files- .gitattributes +24 -0
- NinjaMouse-3B-40L-danube.i1-IQ1_M.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ1_S.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ2_M.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ2_S.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ2_XS.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ2_XXS.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ3_M.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ3_S.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ3_XS.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ3_XXS.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ4_NL.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-IQ4_XS.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q2_K.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q2_K_S.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q3_K_L.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q3_K_M.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q3_K_S.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q4_0.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q4_1.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q4_K_M.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q4_K_S.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q5_K_M.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q5_K_S.gguf +3 -0
- NinjaMouse-3B-40L-danube.i1-Q6_K.gguf +3 -0
.gitattributes
CHANGED
|
@@ -34,3 +34,27 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
imatrix.dat filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
imatrix.dat filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
NinjaMouse-3B-40L-danube.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
NinjaMouse-3B-40L-danube.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
NinjaMouse-3B-40L-danube.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
NinjaMouse-3B-40L-danube.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
NinjaMouse-3B-40L-danube.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
NinjaMouse-3B-40L-danube.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
NinjaMouse-3B-40L-danube.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
NinjaMouse-3B-40L-danube.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
NinjaMouse-3B-40L-danube.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
NinjaMouse-3B-40L-danube.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
NinjaMouse-3B-40L-danube.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
NinjaMouse-3B-40L-danube.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
NinjaMouse-3B-40L-danube.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
NinjaMouse-3B-40L-danube.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 51 |
+
NinjaMouse-3B-40L-danube.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
NinjaMouse-3B-40L-danube.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 53 |
+
NinjaMouse-3B-40L-danube.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
NinjaMouse-3B-40L-danube.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 55 |
+
NinjaMouse-3B-40L-danube.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
| 56 |
+
NinjaMouse-3B-40L-danube.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 57 |
+
NinjaMouse-3B-40L-danube.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 58 |
+
NinjaMouse-3B-40L-danube.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 59 |
+
NinjaMouse-3B-40L-danube.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 60 |
+
NinjaMouse-3B-40L-danube.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
NinjaMouse-3B-40L-danube.i1-IQ1_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:99a1238ff5411f18c85b8928ff6cb8ca845899bf731d1dbbe457c52233453dce
|
| 3 |
+
size 735073088
|
NinjaMouse-3B-40L-danube.i1-IQ1_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7679bbc1b955b7f54d4756ed06843b19811a042eb8a7c24476d3fb3cdca59303
|
| 3 |
+
size 679700288
|
NinjaMouse-3B-40L-danube.i1-IQ2_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f8ef93d2aa517492f5a6c4405391d95d36282d41fc0e704e52e820b6057d0050
|
| 3 |
+
size 1037383488
|
NinjaMouse-3B-40L-danube.i1-IQ2_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3cec87ee47427d6c6fe50a3977ba46256eeb86988c02840e177d592d03bb9ad5
|
| 3 |
+
size 963553088
|
NinjaMouse-3B-40L-danube.i1-IQ2_XS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:768dee1d8638129af278baadacc8941f1f0028a9a158b1beba6f4410890e387e
|
| 3 |
+
size 909383488
|
NinjaMouse-3B-40L-danube.i1-IQ2_XXS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:85d961543247f9a42bafbd75fccc4aa7258c51f5f2213442c0363dad4af67040
|
| 3 |
+
size 827361088
|
NinjaMouse-3B-40L-danube.i1-IQ3_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cfe5a30a12d2808fcef03dc725b45bfa45c58d643d09d2654a8564ad914c0de2
|
| 3 |
+
size 1353236288
|
NinjaMouse-3B-40L-danube.i1-IQ3_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5212c9cf211cbb90417490da875cd935855004db3d7982b1c3824cc179e5f21c
|
| 3 |
+
size 1306669888
|
NinjaMouse-3B-40L-danube.i1-IQ3_XS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b27f42d164394ae9036eaba27a9ce5d2690fc887792fae64a169f0b9b61c1a3d
|
| 3 |
+
size 1241543488
|
NinjaMouse-3B-40L-danube.i1-IQ3_XXS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:275e5304f4baf4b2abeaa9cccb77f39bb2b0a1b72dd5eebca81cf7e58a1a4801
|
| 3 |
+
size 1160417088
|
NinjaMouse-3B-40L-danube.i1-IQ4_NL.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:502005414fff638f71be6a7decaa9c697a5ea836ee7ae4133556a7f343e1b4a5
|
| 3 |
+
size 1686087488
|
NinjaMouse-3B-40L-danube.i1-IQ4_XS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fdf898fa036f08a9cf2c2d184ff9047c8e62e7d85542661dc9bbb6b8710ea915
|
| 3 |
+
size 1598740288
|
NinjaMouse-3B-40L-danube.i1-Q2_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aab8d318b980a166a086f75feea210612f86679912317fad4756a7e5fc1a0a8d
|
| 3 |
+
size 1121300288
|
NinjaMouse-3B-40L-danube.i1-Q2_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5baec566e2fdc133634f2c57d143473f75010af645360fd2d473f4016ac9739b
|
| 3 |
+
size 1043527488
|
NinjaMouse-3B-40L-danube.i1-Q3_K_L.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:62910cd401354b595680f4911b4db5ece91d275e20044693d5401b0c213068c3
|
| 3 |
+
size 1564922688
|
NinjaMouse-3B-40L-danube.i1-Q3_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:936ca57ffb3e92b7bc958e71c73d5ef86773a9d265db0685a08192e935144087
|
| 3 |
+
size 1440322368
|
NinjaMouse-3B-40L-danube.i1-Q3_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6dd2658e006fc9a5e2d7b5a9a3416a5783e015d9396994083b4b5a32c7fd5e61
|
| 3 |
+
size 1297965888
|
NinjaMouse-3B-40L-danube.i1-Q4_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:07906bf3ce4281c774523687b57657cf0965bc14321713bec84c4ff28885b1ac
|
| 3 |
+
size 1683425088
|
NinjaMouse-3B-40L-danube.i1-Q4_1.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2eafb7fb775fdf4c57d05cedb59d6188534a02b3e7b7fe626e8eee8f2adff85e
|
| 3 |
+
size 1856685888
|
NinjaMouse-3B-40L-danube.i1-Q4_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b4bd4a68f742de63357362c9a958488c7f6b195ae410aa2827ace4c48dc22ce0
|
| 3 |
+
size 1777581888
|
NinjaMouse-3B-40L-danube.i1-Q4_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e3f41c0aeaf88afd4cd9b4331cf5a5ba62655380462cef8ce37c71c8a17500b0
|
| 3 |
+
size 1689773888
|
NinjaMouse-3B-40L-danube.i1-Q5_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5a3485f330d7b60356eceffb8419d005220ee0884bd40792793f5319a01aaec1
|
| 3 |
+
size 2086829888
|
NinjaMouse-3B-40L-danube.i1-Q5_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6872b0435125c6adabf015ee18c88929d5dfd20685ed102ed8b206e4386aa3b1
|
| 3 |
+
size 2035476288
|
NinjaMouse-3B-40L-danube.i1-Q6_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a3f07dc1af7122775052328f52d58bfd44e2b53a5be5ce29cba28b741edb6a00
|
| 3 |
+
size 2415405888
|