CineAI/Llama32-3B-CoT-Camel-Unsloth

This model is a fine-tuned version of unsloth/Llama-3.2-3B-bnb-4bit and was fine-tuned using a proprietary CoT dataset created with Camel AI, with Unsloth used to fine-tuned the model.

Training config

• LoRA Rank 16
• Max sequence length 2048
• Max steps 60
• Learning rate 2e-4

Limitations

• The model has limitations due to the small size of the model and the small size of the dataset

Way to use model

You can use the model through the Unsloth FastLanguageModel interface, which boosts the model execution speed by 2 times

    model, tokenizer = FastLanguageModel.from_pretrained(
        model_name     = "model",
        max_seq_length = 2048,
        dtype          = dtype,
        load_in_4bit   = load_in_4bit,
    )

    alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
    ## Instruction:
    {}
    
    ## Input:
    {}
    
    ## Response:
    {}
    """

    FastLanguageModel.for_inference(model)

    inputs = tokenizer(
        [
            alpaca_prompt.format(
                test_questions[0],
                "",
                "",
            )
        ],
        return_tensors="pt"
    ).to("cuda")
    
    outputs = model.generate(
        **inputs,
        max_new_tokens=4096,
        use_cache=True
    )
    
    decoded_outputs = tokenizer.batch_decode(outputs, skip_special_tokens=True)
    print(decoded_outputs[0])

Uploaded model

• Developed by: CineAI
• License: apache-2.0
• Finetuned from model : unsloth/Llama-3.2-3B-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.