Instructions to use arcee-ai/Llama-3-SEC-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use arcee-ai/Llama-3-SEC-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="arcee-ai/Llama-3-SEC-Base")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("arcee-ai/Llama-3-SEC-Base")
model = AutoModelForCausalLM.from_pretrained("arcee-ai/Llama-3-SEC-Base", device_map="auto")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use arcee-ai/Llama-3-SEC-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "arcee-ai/Llama-3-SEC-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Llama-3-SEC-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/arcee-ai/Llama-3-SEC-Base

SGLang

How to use arcee-ai/Llama-3-SEC-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "arcee-ai/Llama-3-SEC-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Llama-3-SEC-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "arcee-ai/Llama-3-SEC-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Llama-3-SEC-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use arcee-ai/Llama-3-SEC-Base with Docker Model Runner:
```
docker model run hf.co/arcee-ai/Llama-3-SEC-Base
```

Crystalcareai commited on Jun 19, 2024

Commit

d6d28e6

verified ·

1 Parent(s): 5c19769

Update README.md

Browse files

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -15,7 +15,7 @@ datasets:
 # Llama-3-SEC-Base: A Domain-Specific Chat Agent for SEC Data Analysis
-Llama-3-SEC-Base is a state-of-the-art domain-specific large language model trained on a vast corpus of SEC (Securities and Exchange Commission) data. Built upon the powerful Meta-Llama-3-70B-Instruct model, Llama-3-SEC-Base has been developed to provide unparalleled insights and analysis capabilities for financial professionals, investors, researchers, and anyone working with SEC filings and related financial data. This checkpoint did not include SFT, and is strictly the model post pretraining that has been merged with Llama-3-70B-Instruct. For a variant that was fine-tuned for chat related purposes please see [Llama-3-SEC-Chat](https://huggingface.co/arcee-ai/Llama-3-SEC-Chat).
 ## Model Details

 # Llama-3-SEC-Base: A Domain-Specific Chat Agent for SEC Data Analysis
+Llama-3-SEC-Base is a state-of-the-art domain-specific large language model trained on a vast corpus of SEC (Securities and Exchange Commission) data. Built upon the powerful Meta-Llama-3-70B-Instruct model, Llama-3-SEC-Base has been developed to provide unparalleled insights and analysis capabilities for financial professionals, investors, researchers, and anyone working with SEC filings and related financial data. This checkpoint does not include supervised fine-tuning (SFT) and is strictly the our CPT model merged with Llama-3-70B-Instruct. For a variant that has been fine-tuned for chat-related purposes, please see [Llama-3-SEC-Chat](https://huggingface.co/arcee-ai/Llama-3-SEC-Chat).
 ## Model Details