README.md

---
base_model: unsloth/qwen3-4b-Instruct-2507-unsloth-bnb-4bit
license: apache-2.0
language:
- en
library_name: peft
tags:
- qwen
- bitcoin
- finance
- text-generation
- lora
- unsloth
- autotrain
datasets:
- tahamajs/not_clearned_bitcoin_dataset
---

Model Card for Qwen-Bitcoin-Chat
This model is a fine-tuned version of unsloth/qwen3-4b-Inststuct-2507-unsloth-bnb-4bit, specialized for predicting Bitcoin prices based on contextual financial data. It has been trained to act as an expert financial analyst, processing historical prices, news, and market metrics to forecast the next 10 days of Bitcoin's closing price.

Model Details
Model Description
This model is a powerful yet efficient language model fine-tuned using the Unsloth library for PEFT (Parameter-Efficient Fine-Tuning) with LoRA. It is designed to understand a complex prompt containing a system message, historical context, and real-time data points (like news headlines, social media sentiment, and other financial indicators).

The model's primary function is to generate a structured response that includes:

A brief analysis of the provided market data.

A comma-separated list of 10 price predictions for the next 10 days.

Special tokens (<|response|>, <|analysis|>, <|forecast|>) are used to structure the output, making it easy to parse for downstream applications.

Developed by: tahamajs

Model type: Causal Language Model based on the Qwen architecture.

Language(s) (NLP): English

License: Apache 2.0

Finetuned from model: unsloth/qwen3-4b-Instruct-2507-unsloth-bnb-4bit

Model Sources
Repository: https://huggingface.co/tahamajs/qwen-bitcoin-chat-manual-Qwen3-4B-Instruct-2507

Uses
Direct Use
This model is intended for generating Bitcoin price forecasts based on provided data. It can be used directly via the transformers library pipeline or by manually creating the prompt. The model expects a specific chat format, as shown in the example below.

Important: This model is for informational and research purposes only. It is not financial advice. Cryptocurrency markets are highly volatile, and predictions should not be used for making investment decisions without consulting a qualified financial advisor.

Downstream Use
This model can be integrated into various applications, such as:

Financial Dashboards: To provide automated daily market commentary and price outlooks.

Trading Bots: As a signal generator, where its analysis and forecasts can be one of several inputs for an automated trading strategy.

Research Tools: To study the relationship between news sentiment, market data, and price movements.

Out-of-Scope Use
Do not use this model as a sole source for financial decisions. Its predictions are based on patterns in the training data and are not guaranteed to be accurate.

The model is not designed for general conversation and may perform poorly on tasks outside of financial analysis and prediction.

The model should not be used to generate misleading or harmful financial content.

Bias, Risks, and Limitations
Data Bias: The model's predictions are heavily influenced by the historical data it was trained on (tahamajs/not_clearned_bitcoin_dataset). It may not perform well in unprecedented market conditions or "black swan" events.

No Real-time Knowledge: The model does not have access to real-time information beyond what is provided in the prompt. Its predictions are only as good as the data it receives.

Hallucination: Like all language models, it can generate information that is plausible-sounding but factually incorrect. The "Analysis" section should be fact-checked if used for any serious purpose.

Volatility Risk: The cryptocurrency market is extremely volatile. The model cannot predict sudden market crashes or spikes caused by unforeseen global events.

Recommendations
Users should treat the model's output as a supplementary tool, not as a definitive forecast. Always combine its analysis with other research and professional advice before making any financial commitments.

How to Get Started with the Model
Use the code below to load the model and run inference. Make sure you have the necessary libraries installed (transformers, torch, accelerate, bitsandbytes, peft).

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tahamajs/qwen-bitcoin-chat-manual-Qwen3-4B-Instruct-2507"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto"
)

# --- 1. Prepare Your Input Data ---
# This should match the format used during training.
instruction = "You are an expert financial analyst. Your primary task is to predict the next 10 days of Bitcoin prices. Analyze the provided news and historical price data to make your forecast."

user_input = """
**Today's Key News:**
['Some recent news snippet here...', 'Another relevant news article...']

**Last 60 Days of Bitcoin's Closing Prices:**
[11323.2, 11657.2, ..., 10106.3, 10221.1]

Use the additional daily data provided in the input below for crucial context.
"""

daily_context = """
Daily Context for Date: 2018-02-01

**Financial & Commodity Data:**
- Gold Closing Price: $1344.30
- Crude Oil Closing Price: $65.80

**Bitcoin Market & On-Chain Metrics:**
- Market Capitalization: $197,280,335,038
- Hash Rate: 20703947.9136968
- Transaction Count: 257664.0
- Unique Addresses: 591551.0

**Social & AI Sentiment:**
- Fear & Greed Index: 0.30
"""

post_input_directive = "Please analyze it first and then give me 10 next day prices separated by comma."

# --- 2. Format the Prompt ---
# Use the chat template defined for the model
messages = [
    {"role": "system", "content": instruction},
    {"role": "user", "content": f"{user_input}\n\n{daily_context}\n\n{post_input_directive}"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

# --- 3. Generate the Prediction ---
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, use_cache=True)
decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=False)

# The full output will be in the chat format.
# You can parse it to extract the analysis and forecast.
print(decoded_output)

# Example of parsing the output
response_start = decoded_output.find("<|response|>")
if response_start != -1:
    response_content = decoded_output[response_start:]
    print("\n--- Parsed Response ---")
    print(response_content)

Training Details
Training Data
The model was fine-tuned on the tahamajs/not_clearned_bitcoin_dataset dataset. This dataset contains historical Bitcoin data, including price, volume, news headlines, and other market indicators. Each entry was formatted into a conversational prompt structure to teach the model to act as a financial analyst.

Training Procedure
The model was trained using the SFTTrainer from the TRL library, which is optimized for instruction fine-tuning.

Preprocessing
The raw data was transformed into a chat format using a custom function (format_chat). This function structures the input with system, user, and assistant roles. Special tokens (<|response|>, <|analysis|>, <|forecast|>) were added to the tokenizer to delineate the different parts of the model's expected output.

A custom DataCollatorMaskResponse was used to ensure that the model is only trained to predict the assistant's response, masking out the user and system prompts from the loss calculation. This focuses the learning process on generating the desired analysis and forecast.

Training Hyperparameters
Framework: PyTorch

Library: Unsloth for PEFT (LoRA)

Base Model: unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit

LoRA Rank (r): 32

LoRA Alpha (lora_alpha): 32

Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Epochs: 4

Max Steps: 400

Per Device Train Batch Size: 4

Gradient Accumulation Steps: 1

Optimizer: AdamW (adamw_torch)

Learning Rate: 2e-4

LR Scheduler: Cosine

Warmup Ratio: 0.05

Precision: fp16

Framework versions
Transformers 4.41.2

Pytorch 2.3.0+cu121

Datasets 2.19.1

PEFT 0.11.1

Unsloth 2024.5