Instructions to use aisingapore/SEA-LION-v1-7B-IT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aisingapore/SEA-LION-v1-7B-IT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aisingapore/SEA-LION-v1-7B-IT", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("aisingapore/SEA-LION-v1-7B-IT", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("aisingapore/SEA-LION-v1-7B-IT", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aisingapore/SEA-LION-v1-7B-IT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aisingapore/SEA-LION-v1-7B-IT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/SEA-LION-v1-7B-IT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/aisingapore/SEA-LION-v1-7B-IT

SGLang

How to use aisingapore/SEA-LION-v1-7B-IT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aisingapore/SEA-LION-v1-7B-IT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/SEA-LION-v1-7B-IT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aisingapore/SEA-LION-v1-7B-IT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/SEA-LION-v1-7B-IT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use aisingapore/SEA-LION-v1-7B-IT with Docker Model Runner:
```
docker model run hf.co/aisingapore/SEA-LION-v1-7B-IT
```

SEA-LION-v1-7B-IT

Commit History

Fix typo in chat template

52168f9

xianbin commited on Jun 5, 2024

Add chat template in tokenizer_config.json

b667b1c
verified

jahhs0n commited on Jun 5, 2024

Update metrics in README.md

566afff
verified

weiqipedia commited on Apr 12, 2024

Fix the bug in generation config (#2)

cab651c
verified

RaymondAISG

SivilTaram commited on Apr 11, 2024

Update tokenizer.model for GGUF quantization

5c84557
verified

xianbin commited on Apr 3, 2024

Update README.md

07d2e1f
verified

RaymondAISG commited on Apr 1, 2024

Update README.md

68b25b8
verified

RaymondAISG commited on Mar 16, 2024

Update README.md

a7bb812
verified

xianbin commited on Mar 5, 2024

Updated readme to add more details on bhasa

5282321
verified

tainc commited on Mar 5, 2024

update links to new naming scheme

a28abe3
verified

tainc commited on Mar 4, 2024

Update instruct model to latest weights

881b143

Xianbin commited on Mar 4, 2024

Update README.md

83f8193
verified

holylovenia commited on Feb 29, 2024

Delete model-00002-of-00002.safetensors

c6121d8
verified

xianbin commited on Feb 16, 2024

Delete model-00001-of-00002.safetensors

d9e525f
verified

xianbin commited on Feb 16, 2024

Update model weights to latest version

60b568e
verified

xianbin commited on Feb 16, 2024

Update README.md

10f60d3
verified

xianbin commited on Feb 16, 2024

Add model

3f96a16

holylovenia commited on Feb 1, 2024

Update README.md

678e9d8
verified

holylovenia commited on Feb 1, 2024

Update README.md

37c45b5
verified

holylovenia commited on Feb 1, 2024

initial commit

52de312
verified

holylovenia commited on Feb 1, 2024

Commit History

Fix typo in chat template 52168f9

Add chat template in tokenizer_config.json b667b1c verified

Update metrics in README.md 566afff verified

Fix the bug in generation config (#2) cab651c verified

Update tokenizer.model for GGUF quantization 5c84557 verified

Update README.md 07d2e1f verified

Update README.md 68b25b8 verified

Update README.md a7bb812 verified

Updated readme to add more details on bhasa 5282321 verified

update links to new naming scheme a28abe3 verified

Update instruct model to latest weights 881b143

Update README.md 83f8193 verified

Delete model-00002-of-00002.safetensors c6121d8 verified

Delete model-00001-of-00002.safetensors d9e525f verified

Update model weights to latest version 60b568e verified

Update README.md 10f60d3 verified

Add model 3f96a16

Update README.md 678e9d8 verified

Update README.md 37c45b5 verified

initial commit 52de312 verified

Fix typo in chat template

52168f9

Add chat template in tokenizer_config.json

b667b1c
verified

Update metrics in README.md

566afff
verified

Fix the bug in generation config (#2)

cab651c
verified

Update tokenizer.model for GGUF quantization

5c84557
verified

Update README.md

07d2e1f
verified

Update README.md

68b25b8
verified

Update README.md

a7bb812
verified

Updated readme to add more details on bhasa

5282321
verified

update links to new naming scheme

a28abe3
verified

Update instruct model to latest weights

881b143

Update README.md

83f8193
verified

Delete model-00002-of-00002.safetensors

c6121d8
verified

Delete model-00001-of-00002.safetensors

d9e525f
verified

Update model weights to latest version

60b568e
verified

Update README.md

10f60d3
verified

Add model

3f96a16

Update README.md

678e9d8
verified

Update README.md

37c45b5
verified

initial commit

52de312
verified