Instructions to use aisingapore/SEA-LION-v1-7B-IT-Research with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aisingapore/SEA-LION-v1-7B-IT-Research with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aisingapore/SEA-LION-v1-7B-IT-Research", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("aisingapore/SEA-LION-v1-7B-IT-Research", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("aisingapore/SEA-LION-v1-7B-IT-Research", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aisingapore/SEA-LION-v1-7B-IT-Research with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aisingapore/SEA-LION-v1-7B-IT-Research"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/SEA-LION-v1-7B-IT-Research",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/aisingapore/SEA-LION-v1-7B-IT-Research

SGLang

How to use aisingapore/SEA-LION-v1-7B-IT-Research with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aisingapore/SEA-LION-v1-7B-IT-Research" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/SEA-LION-v1-7B-IT-Research",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aisingapore/SEA-LION-v1-7B-IT-Research" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/SEA-LION-v1-7B-IT-Research",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use aisingapore/SEA-LION-v1-7B-IT-Research with Docker Model Runner:
```
docker model run hf.co/aisingapore/SEA-LION-v1-7B-IT-Research
```

Bahasa Translation

by becks2000 - opened Mar 19, 2024

Discussion

becks2000

Mar 19, 2024

Hi
Have your tried with bahasa translation? Do your have some score using this model to do translation?

Thanks

weiqipedia

AI Singapore org Apr 1, 2024

Hi thank you very much for your inquiry! If the bahasa that you are referring to is Bahasa Indonesia, we do have some scores for the task of machine translation. We have tested it on the FLoRes-200 dataset in both directions (Indonesian to English & English to Indonesian) and the scores can be found in the model card as well.

The prompts used are as follows:

# For English to Indonesian
Terjemahkan teks berikut ini ke dalam Bahasa Indonesia.\nTeks: {text}\nTerjemahan:  

# For Indonesian to English
Terjemahkan teks berikut ini ke dalam Bahasa Inggris.\nTeks: {text}\nTerjemahan:

With this Indonesian prompt, sea-lion-7b-instruct-research obtained a ChrF++ score of 52.50 for English to Indonesian and 46.82 for Indonesian to English.

We also have a newer model, sea-lion-7b-instruct, which has an improved ChrF++ score of 57.48 for English to Indonesian and 58.04 for Indonesian to English.

You may also find comparisons with other 7B models in the model card. (Screenshot attached here)

I hope this addresses your question!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment