---
title: Q-TensorFormer
emoji: ⚛️
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 4.44.1
app_file: app.py
pinned: false
license: apache-2.0
---

# Q-TensorFormer: Quantum-Enhanced Tensor Network LLM Compression Engine

## Overview

**Q-TensorFormer** is a hybrid quantum-tensor model that adaptively compresses itself using entanglement entropy, achieving major efficiency gains with minimal performance loss.

**Claim**: 50-70% parameter reduction with same accuracy ± small drop, fewer compute ops / latency.

## Architecture

### Three Pillars

1. **Tensor Compression (Efficiency)**
   - Dense FFN layers replaced with Tensor-Train (TT) decomposition via tltorch
   - Dramatic parameter reduction while preserving expressivity

2. **Quantum Feature Encoding (Expressivity)**
   - PennyLane quantum circuits encode token embeddings into quantum states
   - Angle encoding + variational circuits extract richer features than classical

3. **Entanglement-Guided Rank Adaptation (Novelty)**
   - `r = r_min + α · S(ρ)` — tensor ranks adjust based on quantum state entropy
   - Model becomes input-aware and compute-efficient

### Core Components

- `TTFactorizedLinear`: Tensor-Train compressed linear layers
- `QuantumFeatureEncoder`: PennyLane angle encoding with TorchLayer
- `QuantumKernelAttention`: Quantum kernel self-attention (QKSAN-style)
- `SelectiveQuantumRouter`: Only "hard" tokens go to quantum circuit
- `RankScheduler`: Entanglement-guided dynamic rank adjustment

## Results

| Metric | Baseline | Q-TensorFormer | Reduction |
|--------|----------|----------------|-----------|
| Parameters | 10,764,288 | 1,325,102 | **8.12x** |
| Memory (MB) | ~42 MB | ~5 MB | **8.12x** |
| Compression | 1.00x | 8.12x | ✓ |

## Usage

```python
from qtensorformer import QTensorFormer, ModelConfig

config = ModelConfig(
    vocab_size=10000,
    hidden_dim=128,
    n_layers=3,
    tt_rank=4,
    n_qubits=4,
    use_quantum_attention=True,
    use_adaptive_rank=True,
)

model = QTensorFormer(config)
logits, loss, stats = model(input_ids, labels=labels)
```

## Citation

```bibtex
@misc{qtensorformer2025,
  title={Q-TensorFormer: Quantum-Enhanced Tensor Network LLM Compression},
  author={Q-TensorFormer Team},
  year={2025},
  note={Hybrid quantum-tensor model with entanglement-guided compression}
}
```

## References

- QKSAN (Quantum Kernel Self-Attention Network): arXiv:2308.13422
- tltorch: TensorLy-Torch for deep tensor learning
- PennyLane: Quantum machine learning library

## Final Evaluation Results (WikiText-2)

| Metric | Baseline (Dense) | Q-TensorFormer |
|--------|------------------|----------------|
| Parameters | 1,554,570 | 793,882 |
| **Compression** | **1.00x** | **2.0x** |
| BlockTT Active | — | ✓ |
| Adaptive Rank Range | — | 2–3 (mean: 3.0) |
| Entanglement Range | — | 0.855–1.666 |
| Quantum Routing Savings | — | 80% |

### Key Findings

1. **BlockTT decomposition** provides 2.0x parameter compression on WikiText-2
2. **Entanglement entropy varies** across real tokens (0.855–1.666), enabling per-token adaptation
3. **Adaptive rank changes** from 2 to 3 based on token complexity via r = r_min + α·S(ρ)
4. **Selective quantum routing** saves 80% of quantum circuit evaluations
5. **K2 Think integration** provides explainable AI for rank and routing decisions

### Explainable AI

The model uses K2 Think (MBZUAI-IFM/K2-Think-v2) to generate natural language
explanations for every compression and routing decision, making tensor network
compression transparent and auditable.