Instructions to use SURESHBEEKHANI/Llama-2-7b-chat-finetune with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SURESHBEEKHANI/Llama-2-7b-chat-finetune with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SURESHBEEKHANI/Llama-2-7b-chat-finetune")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SURESHBEEKHANI/Llama-2-7b-chat-finetune")
model = AutoModelForCausalLM.from_pretrained("SURESHBEEKHANI/Llama-2-7b-chat-finetune", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SURESHBEEKHANI/Llama-2-7b-chat-finetune with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SURESHBEEKHANI/Llama-2-7b-chat-finetune"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SURESHBEEKHANI/Llama-2-7b-chat-finetune",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SURESHBEEKHANI/Llama-2-7b-chat-finetune

SGLang

How to use SURESHBEEKHANI/Llama-2-7b-chat-finetune with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SURESHBEEKHANI/Llama-2-7b-chat-finetune" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SURESHBEEKHANI/Llama-2-7b-chat-finetune",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SURESHBEEKHANI/Llama-2-7b-chat-finetune" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SURESHBEEKHANI/Llama-2-7b-chat-finetune",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SURESHBEEKHANI/Llama-2-7b-chat-finetune with Docker Model Runner:
```
docker model run hf.co/SURESHBEEKHANI/Llama-2-7b-chat-finetune
```

YAML Metadata Warning:The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other

Model Card for Llama-2-7b-chat-finetune

This model is a fine-tuned version of Llama-2-7b for chat-based applications, trained on medical data to answer various queries with detailed medical knowledge.

Model Details

Model Description

This model is fine-tuned from Llama-2-7b for answering medical-related queries and tasks using a large corpus of medical data. It is suitable for generating text based on a given prompt in a conversational style.

Developed by: SURESHBEEKHANI
License: MIT
Model type: Causal Language Model
Language(s): English
Finetuned from model: Llama-2-7b

Model Sources

Repository: SURESHBEEKHANI/Llama-2-7b-chat-finetune
Code Notebook: Fine-tune Llama-2-7b

Use Cases

Direct Use

This model can be used directly for generating text responses to prompts related to medical topics. It is designed to assist in answering medical queries with detailed information.

Out-of-Scope Use

This model is not suitable for generating answers related to non-medical domains, and should not be used in contexts where the data might be sensitive, harmful, or biased.

Bias, Risks, and Limitations

The model might inherit biases from its training data and might not always provide accurate medical information. It is recommended to use the model as a supplementary tool and consult medical professionals for critical use cases.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import pipeline, AutoModelForCausalLM, AutoTokenizer

model_name = "SURESHBEEKHANI/Llama-2-7b-chat-finetune"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = " What is Superficial vein thrombosis and explain in detail? ?"
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=200)
result = pipe(f"<s>[INST] {prompt} [/INST]")
print(result[0]['generated_text'])

Downloads last month: 6

Model tree for SURESHBEEKHANI/Llama-2-7b-chat-finetune

Base model

NousResearch/Llama-2-7b-chat-hf

Finetuned

(142)

this model

SURESHBEEKHANI
/

Llama-2-7b-chat-finetune