Instructions to use mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF with Ollama:
```
ollama run hf.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M
```

Unsloth Studio

How to use mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF to start chatting

Docker Model Runner
How to use mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.OpenMia-Indo-Mistral-7b-v3-i1-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

mradermacher commited on Dec 22, 2024

Commit

0f428d8

verified ·

1 Parent(s): 9ca34cc

uploaded from rain

Browse files

Files changed (23) hide show

.gitattributes +22 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf +3 -0
OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,25 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 imatrix.dat filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 imatrix.dat filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7d9fe9bfe7eda3c1916e22d6e98a7f0d049f3449690f6996f5e1fd81f011a842
+size 1754447648

OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9ccc6bd7cf153e2f2017ab08d56b8c7b766063659eb8179a1c1b48f4f3dcc4f3
+size 1612103456

OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6efe1aa4c2fa63aa82cd803acc253e973a2a1713baa4a13dc6edd9eba15df44b
+size 2500714272

OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7d070b28206cf7de537bb8395546c1dbda565f01a43f2d9d3649f49c954d1461
+size 2310922016

OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5dd6146aecbcb49b9a15abd6ecbd6f872eebab60f60e340dc029a63cddca5d21
+size 2198257440

OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:df7d11d27255360ebab386d5b21e4cdb8b8788a1113d187a77045768ae7ec94c
+size 1991687968

OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0aea357149a54f25ad27f1054dfe40e61a64b3f14ceac537fee815a21febb848
+size 3284893472

OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5dcb659c7b181652197a397b6aea7042fe30403acbbfcd4f443acce658ba4527
+size 3182395168

OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:826b59d887a828a2c02a6a56404b16250ab65f36b92eecf6d43139e9784521dd
+size 3018817312

OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:387f10f92d8e9f29e7e47bf2b4bb28a69614f325c060f650e324e6614ef4b24c
+size 2827345696

OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c33b77bd0e32c82a1ea1d9a5f4653a2213d1cf61df707123f60e30dbe34ae55f
+size 3907690272

OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f53563db36a797b7da26f2b00cbcf64296a25a98db7cb76c8b458e247d683f37
+size 2719244064

OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9ad329e166472e137f87626e18569e0cc329976ca53bc745a10a3c2a3f3782c3
+size 2528927520

OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c5149563fe8e1d86e9a1f01bbb568ace3a49067785f0a7fe5a06d4d131e0a1a7
+size 3822026528

OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:96d96f41c15ce721c130feace59ceeac40c45f96480c8cc7d5c7da09557b36a7
+size 3518988064

OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1885d6a12e81a8bd5602df4c22efe997ed573064e25979a709a8cfaae98469f1
+size 3164569376

OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a32ef64ac0b5e1514c3f07e4b0336b12722a8f1a5c38a7b8ce19a5127f6a828b
+size 4123598624

OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b166cff0b30b7d5c4c6a9be318e27ea5e950f7c7b7c3d793354ffd8fed03248e
+size 4368441120

OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:be01780f5ea76e8320ea0a891a6c76d8a57958af13bacf7c421126bc63563a48
+size 4140375840

OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:da0b2fcc52ac3930e1e425d124789c7b827553c5e297274e2880a09bffc37132
+size 5131411232

OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3fdb2184e44669a5f5f9b2c22e54575f72bc9a94b9ff2bf1ca6b4cfcf8b8a9bc
+size 4997717792

OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ee976a0d005ed0c1834ee105d6b9f35459f81c94470dce41e38c7cfa9a4c0eba
+size 5942066976