Instructions to use Vikhrmodels/Vikhr-Gemma-2B-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Vikhrmodels/Vikhr-Gemma-2B-instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Vikhrmodels/Vikhr-Gemma-2B-instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Vikhrmodels/Vikhr-Gemma-2B-instruct") model = AutoModelForCausalLM.from_pretrained("Vikhrmodels/Vikhr-Gemma-2B-instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Vikhrmodels/Vikhr-Gemma-2B-instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Vikhrmodels/Vikhr-Gemma-2B-instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vikhrmodels/Vikhr-Gemma-2B-instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Vikhrmodels/Vikhr-Gemma-2B-instruct
- SGLang
How to use Vikhrmodels/Vikhr-Gemma-2B-instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Vikhrmodels/Vikhr-Gemma-2B-instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vikhrmodels/Vikhr-Gemma-2B-instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Vikhrmodels/Vikhr-Gemma-2B-instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vikhrmodels/Vikhr-Gemma-2B-instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Vikhrmodels/Vikhr-Gemma-2B-instruct with Docker Model Runner:
docker model run hf.co/Vikhrmodels/Vikhr-Gemma-2B-instruct
Incompatible license: Apache-2.0 License is Incompatible with Gemma License
Hi, I'd like to report a License Conflict in Vikhrmodels/Vikhr-Gemma-2B-instruct. I noticed this model was fintuned from google/gemma-2-2b-it, which is released under the Gemma license. From what I can see, Vikhrmodels/Vikhr-Gemma-2B-instruct appears to be incompatible with Gemma’s clauses — especially regarding redistribution, sublicensing, and commercial use.
⚠️ Key violations of Gemma license:
Section 3.1 – Distribution and Redistribution:
• Redistributing a derivative (like this one) requires including a copy of the Gemma license
• Must include a "NOTICE" file with this statement:
"Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms"
• Must carry over the use restrictions from Section 3.2 (Google’s Prohibited Use Policy)
• Any additional license terms (like Apache-2.0) must NOT conflict with the Gemma License
Section 3.2 – Use Restrictions:
• Must not use the model for any prohibited purposes
• Must comply with applicable laws and Google’s Prohibited Use Policy
Section 2.2 – Use Terms:
• Usage is only allowed “in accordance with the Gemma Terms of Use”
Meanwhile, Apache-2.0 allows:
• Sublicensing and redistribution under permissive terms
• No requirement to propagate upstream non-permissive terms or use restrictions
This creates a conflict because Gemma’s license explicitly prohibits sublicensing under more permissive terms and requires downstream users to inherit specific use restrictions — something Apache-2.0 does not enforce.
🔹 Suggestions (friendly ideas to help bring things in line! ):
1. To better align with Gemma’s license terms, it could help to revise the current license setup a bit. For example:
- Include a full copy of the Gemma License in the repository or model card
-Add a required "NOTICE" file with this statement:
> “Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms”
-Making it clear in the model card that this is a derivative of a Gemma model, and that it inherits the same use restrictions (e.g., no commercial use, no sublicensing, etc.)
2. It might be a good idea to remove the Apache-2.0 license tag if the model is strictly derived from Gemma — since Apache-2.0’s terms can conflict with Gemma’s restrictions. Switching to just the Gemma license could help avoid confusion for downstream users who want to stay compliant.
Thanks for your attention!
Would love to hear your view on this!