README.md

---
title: Q-TensorFormer
emoji: ⚛️
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 4.44.1
app_file: app.py
pinned: false
license: apache-2.0
---

# Q-TensorFormer: Quantum-Enhanced Tensor Network LLM Compression Engine

## Overview

**Q-TensorFormer** is a hybrid quantum-tensor model that adaptively compresses itself using entanglement entropy, achieving major efficiency gains with minimal performance loss.

**Claim**: 50-70% parameter reduction with same accuracy ± small drop, fewer compute ops / latency.

## Architecture

### Three Pillars

1. **Tensor Compression (Efficiency)**
   - Dense FFN layers replaced with Tensor-Train (TT) decomposition via tltorch
   - Dramatic parameter reduction while preserving expressivity

2. **Quantum Feature Encoding (Expressivity)**
   - PennyLane quantum circuits encode token embeddings into quantum states
   - Angle encoding + variational circuits extract richer features than classical

3. **Entanglement-Guided Rank Adaptation (Novelty)**
   - `r = r_min + α · S(ρ)` — tensor ranks adjust based on quantum state entropy
   - Model becomes input-aware and compute-efficient

### Core Components

- `TTFactorizedLinear`: Tensor-Train compressed linear layers
- `QuantumFeatureEncoder`: PennyLane angle encoding with TorchLayer
- `QuantumKernelAttention`: Quantum kernel self-attention (QKSAN-style)
- `SelectiveQuantumRouter`: Only "hard" tokens go to quantum circuit
- `RankScheduler`: Entanglement-guided dynamic rank adjustment

## Results

| Metric | Baseline | Q-TensorFormer | Reduction |
|--------|----------|----------------|-----------|
| Parameters | 10,764,288 | 1,325,102 | **8.12x** |
| Memory (MB) | ~42 MB | ~5 MB | **8.12x** |
| Compression | 1.00x | 8.12x | ✓ |

## Usage

```python
from qtensorformer import QTensorFormer, ModelConfig

config = ModelConfig(
    vocab_size=10000,
    hidden_dim=128,
    n_layers=3,
    tt_rank=4,
    n_qubits=4,
    use_quantum_attention=True,
    use_adaptive_rank=True,
)

model = QTensorFormer(config)
logits, loss, stats = model(input_ids, labels=labels)
```

## Citation

```bibtex
@misc{qtensorformer2025,
  title={Q-TensorFormer: Quantum-Enhanced Tensor Network LLM Compression},
  author={Q-TensorFormer Team},
  year={2025},
  note={Hybrid quantum-tensor model with entanglement-guided compression}
}
```

## References

- QKSAN (Quantum Kernel Self-Attention Network): arXiv:2308.13422
- tltorch: TensorLy-Torch for deep tensor learning
- PennyLane: Quantum machine learning library