Typhoon 1.5
Collection
06-2024 Text ThaiLLM release developed by SCB 10X. • 6 items • Updated
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf typhoon-ai/llama-3-typhoon-v1.5-8b-instruct-gguf:Q4_K_M# Run inference directly in the terminal:
llama cli -hf typhoon-ai/llama-3-typhoon-v1.5-8b-instruct-gguf:Q4_K_MUse pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf typhoon-ai/llama-3-typhoon-v1.5-8b-instruct-gguf:Q4_K_M# Run inference directly in the terminal:
./llama-cli -hf typhoon-ai/llama-3-typhoon-v1.5-8b-instruct-gguf:Q4_K_MBuild from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf typhoon-ai/llama-3-typhoon-v1.5-8b-instruct-gguf:Q4_K_M# Run inference directly in the terminal:
./build/bin/llama-cli -hf typhoon-ai/llama-3-typhoon-v1.5-8b-instruct-gguf:Q4_K_MUse Docker
docker model run hf.co/typhoon-ai/llama-3-typhoon-v1.5-8b-instruct-gguf:Q4_K_MQuick Links
Llama-3-Typhoon-1.5-8B: Thai Large Language Model (Instruct) - GGUF
Llama-3-Typhoon-1.5-8B-instruct is a instruct Thai 🇹🇠large language model with 8 billion parameters, and it is based on Llama3-8B.
Here is gguf converted of Typhoon-1.5-8b-instruct.
Chat Template
We use llama3 chat-template for LM Studio.
{
"name": "Llama 3",
"inference_params": {
"input_prefix": "<|start_header_id|>user<|end_header_id|>\n\n",
"input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
"pre_prompt": "You are a helpful assistant who're always speak Thai.",
"pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\n\n",
"pre_prompt_suffix": "<|eot_id|>",
"antiprompt": [
"<|start_header_id|>", "<|eot_id|>"
]
}
}
Intended Uses & Limitations
This model is an instructional model. However, it’s still undergoing development. It incorporates some level of guardrails, but it still may produce answers that are inaccurate, biased, or otherwise objectionable in response to user prompts. We recommend that developers assess these risks in the context of their use case.
Follow us
https://twitter.com/opentyphoon
Support
SCB10X AI Team
- Kunat Pipatanakul, Potsawee Manakul, Sittipong Sripaisarnmongkol, Natapong Nitarach, Pathomporn Chokchainant, Kasima Tharnpipitchai
- If you find Typhoon-8B useful for your work, please cite it using:
@article{pipatanakul2023typhoon,
title={Typhoon: Thai Large Language Models},
author={Kunat Pipatanakul and Phatrasek Jirabovonvisut and Potsawee Manakul and Sittipong Sripaisarnmongkol and Ruangsak Patomwong and Pathomporn Chokchainant and Kasima Tharnpipitchai},
year={2023},
journal={arXiv preprint arXiv:2312.13951},
url={https://arxiv.org/abs/2312.13951}
}
Contact Us
- General & Collaboration: kasima@scb10x.com, pathomporn@scb10x.com
- Technical: kunat@scb10x.com
- Downloads last month
- 99
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf typhoon-ai/llama-3-typhoon-v1.5-8b-instruct-gguf:Q4_K_M# Run inference directly in the terminal: llama cli -hf typhoon-ai/llama-3-typhoon-v1.5-8b-instruct-gguf:Q4_K_M