README.md

metadata

title: Q-TensorFormer
emoji: ⚛️
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 4.44.1
app_file: app.py
pinned: false
license: apache-2.0

Q-TensorFormer: Quantum-Enhanced Tensor Network LLM Compression Engine

Overview

Q-TensorFormer is a hybrid quantum-tensor model that adaptively compresses itself using entanglement entropy, achieving major efficiency gains with minimal performance loss.

Claim: 50-70% parameter reduction with same accuracy ± small drop, fewer compute ops / latency.

Architecture

Three Pillars

1. Tensor Compression (Efficiency)

   • Dense FFN layers replaced with Tensor-Train (TT) decomposition via tltorch
   • Dramatic parameter reduction while preserving expressivity

2. Quantum Feature Encoding (Expressivity)

   • PennyLane quantum circuits encode token embeddings into quantum states
   • Angle encoding + variational circuits extract richer features than classical

3. Entanglement-Guided Rank Adaptation (Novelty)

   • r = r_min + α · S(ρ) — tensor ranks adjust based on quantum state entropy
   • Model becomes input-aware and compute-efficient

Core Components

• TTFactorizedLinear: Tensor-Train compressed linear layers
• QuantumFeatureEncoder: PennyLane angle encoding with TorchLayer
• QuantumKernelAttention: Quantum kernel self-attention (QKSAN-style)
• SelectiveQuantumRouter: Only "hard" tokens go to quantum circuit
• RankScheduler: Entanglement-guided dynamic rank adjustment

Results

Metric	Baseline	Q-TensorFormer	Reduction
Parameters	10,764,288	1,325,102	8.12x
Memory (MB)	~42 MB	~5 MB	8.12x
Compression	1.00x	8.12x	✓

Usage

from qtensorformer import QTensorFormer, ModelConfig

config = ModelConfig(
    vocab_size=10000,
    hidden_dim=128,
    n_layers=3,
    tt_rank=4,
    n_qubits=4,
    use_quantum_attention=True,
    use_adaptive_rank=True,
)

model = QTensorFormer(config)
logits, loss, stats = model(input_ids, labels=labels)

Citation

@misc{qtensorformer2025,
  title={Q-TensorFormer: Quantum-Enhanced Tensor Network LLM Compression},
  author={Q-TensorFormer Team},
  year={2025},
  note={Hybrid quantum-tensor model with entanglement-guided compression}
}

References

• QKSAN (Quantum Kernel Self-Attention Network): arXiv:2308.13422
• tltorch: TensorLy-Torch for deep tensor learning
• PennyLane: Quantum machine learning library