Text Generation
Transformers
PyTorch
Safetensors
English
gpt2
Trained with AutoTrain
text-generation-inference
Instructions to use CrabfishAI/NeXGen-small with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use CrabfishAI/NeXGen-small with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="CrabfishAI/NeXGen-small")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("CrabfishAI/NeXGen-small", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use CrabfishAI/NeXGen-small with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "CrabfishAI/NeXGen-small" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CrabfishAI/NeXGen-small", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/CrabfishAI/NeXGen-small
- SGLang
How to use CrabfishAI/NeXGen-small with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "CrabfishAI/NeXGen-small" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CrabfishAI/NeXGen-small", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "CrabfishAI/NeXGen-small" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CrabfishAI/NeXGen-small", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use CrabfishAI/NeXGen-small with Docker Model Runner:
docker model run hf.co/CrabfishAI/NeXGen-small
| tags: | |
| - autotrain | |
| - text-generation | |
| widget: | |
| - text: Once upon a time, | |
| - text: My name is john and my hobby is | |
| - text: My hobby was playing cricket but now i | |
| - text: I asked my biology teacher that | |
| - text: I love playing | |
| - text: I came back to home to pet my cat but then | |
| - text: I never received a letter from John Lewis after he | |
| license: mit | |
| language: | |
| - en | |
| # NeXGen - A Text Generative Model | |
| Note- this is the smallest version of NeXGen series we,ll realise larger versions of NeXGen soon stay-tuned. | |
| Based version of NeXGen at: [CrabfishAI/NeXGen-based](https://huggingface.co/CrabfishAI/NeXGen-based) | |
| Large version of NexGen at: [CrabfishAI/NeXGen-large](https://huggingface.co/CrabfishAI/NeXGen-large) | |
| Introduction-NeXGen is a state-of-the-art text generative model designed to meet diverse needs, from creative writing to content creation. This model leverages advanced natural language processing techniques to provide human-like text generation with a wide range of applications. | |
| ## Features | |
| - **Creative Content Generation:** NeXGen excels at generating creative writing, including stories, poetry, and fictional narratives. | |
| - **Contextual Awareness:** The model understands context, ensuring coherent and contextually appropriate responses. | |
| - **User-Friendly Interface:** NeXGen offers an intuitive and user-friendly interface for seamless integration into various applications. | |
| - **Versatility:** From content creation to educational support, NeXGen adapts to different writing styles and applications. | |
| - **Advanced Architecture:** Built on the latest advancements in natural language processing, NeXGen offers high-quality text generation. | |
| ## Uses | |
| NeXGen finds application in various domains, including: | |
| - **Content Creation:** Generate marketing copy, stories, and product descriptions. | |
| - **Assistance in Writing:** Aid authors, bloggers, and students in drafting articles and essays. | |
| - **Chatbot Development:** Power conversational agents with human-like responses. | |
| - **Prototyping and Idea Generation:** Facilitate brainstorming sessions for product development. | |
| - **Social Media Content:** Generate engaging captions for social media posts. | |
| - **Personal Assistant Applications:** Assist users in drafting emails and messages. | |
| ## Direct Use Cases | |
| NeXGen can be directly employed for: | |
| - **Automated Email Drafting:** Quickly compose emails with NeXGen's assistance. | |
| - **Blog Post Generation:** Generate sections or entire articles based on a given topic. | |
| - **Code Commenting:** Improve code documentation with clear and concise comments. | |
| - **Storyline Creation for Games:** Create dynamic and engaging storylines for video games. | |
| - **Learning Material Generation:** Develop study guides and educational content. | |
| - **Personal Journaling Assistance:** Receive prompts and suggestions for journaling. | |
| ## Getting Started | |
| To download NeXGen use this code: | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForCausalLM | |
| # Specify the model name from Hugging Face Model Hub | |
| model_name = "CrabfishAI/NeXGen-small" | |
| tokenizer = AutoTokenizer.from_pretrained(model_name) | |
| model = AutoModelForCausalLM.from_pretrained(model_name) | |
| def generate_text(prompt, max_length=100, num_beams=5, no_repeat_ngram_size=2, top_k=50, top_p=0.95, temperature=0.7): | |
| input_ids = tokenizer.encode(prompt, return_tensors="pt") | |
| # Ensure attention_mask is provided | |
| attention_mask = input_ids.ne(tokenizer.pad_token_id).float() | |
| # Generate output text | |
| output = model.generate( | |
| input_ids, | |
| max_length=max_length, | |
| num_beams=num_beams, | |
| no_repeat_ngram_size=no_repeat_ngram_size, | |
| top_k=top_k, | |
| top_p=top_p, | |
| temperature=temperature, | |
| attention_mask=attention_mask # Pass attention_mask to the generation method | |
| ) | |
| decoded_output = tokenizer.decode(output[0], skip_special_tokens=True) | |
| return decoded_output | |
| # Example usage: | |
| prompt = "Your prompt here" | |
| generated_text = generate_text(prompt, max_length=200) | |
| print("Generated Text:") | |
| print(generated_text) | |
| ``` | |
| ## Limitation | |
| 1. **Content Quality**: The model's output may vary in quality, and there's a possibility it might generate content that is nonsensical, irrelevant, or grammatically incorrect. | |
| 2. **Bias and Sensitivity**: The model is trained on diverse data, but it may inadvertently exhibit biases or generate content that is sensitive or inappropriate. Exercise caution and review generated text before use. | |
| 3. **Inappropriate Language**: The model might generate text that includes offensive language or inappropriate content. Be mindful of this, especially in applications where maintaining a respectful and inclusive tone is essential. | |
| 4. **Ambiguous Prompts**: The quality of generated text is highly dependent on the prompt provided. Ambiguous or unclear prompts may result in less coherent or relevant outputs. | |
| ## Disclaimer | |
| - **Use with Caution**: This model is a tool that should be used with caution. Always review and validate the generated text before incorporating it into any application or publication. | |
| - **Not for Critical Applications**: Avoid using the model for critical applications where accuracy and reliability are paramount. The model is intended for creative and exploratory purposes. | |
| - **Ongoing Improvement**: The model may be updated or fine-tuned for better performance. Stay informed about updates and consider using the latest version for improved results. |