---
license: apache-2.0
pipeline_tag: other
tags:
- tabular
- foundation-model
---

# LimiX-2M

LimiX-2M is a 2M-parameter tabular foundation model designed to mitigate low-rank collapse and attention bottlenecks in structured data. It was introduced in the paper [LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models](https://huggingface.co/papers/2606.04485).

- **GitHub Repository:** [https://github.com/limix-ldm-ai/LimiX](https://github.com/limix-ldm-ai/LimiX)
- **Project Page:** [https://www.limix.ai/](https://www.limix.ai/)

# 1. Model Introduction

**LimiX** is a new class of tabular AI model designed to overcome one of modern machine learning’s longest-standing bottlenecks: structured data. With only **2M parameters**, **LimiX-2M** sets a new state-of-the-art across classification, regression, and missing-value imputation, surpassing XGBoost, CatBoost, AutoGluon, and TabPFN. Its lightweight, training-free design makes advanced tabular modeling accessible on ordinary hardware while preserving full transparency and offline deployability.

**Key Features**

- **Unified Tabular Reasoning:** End-to-end designed for multi-task tabular intelligence, enabling a single model to handle classification, regression, and imputation without additional tuning or task-specific fine-tuning.
- **Training-Free, Context-Driven Inference:** Operates directly through in-context learning: no training, no hyperparameters, no preprocessing pipelines. 
- **Lightweight & Efficient Deployment:** A compact 2M-parameter architecture enables fast inference on standard CPUs and laptops.

# 2. Model Usage

To use LimiX-2M, you need to clone the official repository and install the dependencies listed in the [Deployment](#4-deployment) section.

### Classification Task Example

```python
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import accuracy_score, roc_auc_score
from sklearn.model_selection import train_test_split
from huggingface_hub import hf_hub_download
import numpy as np
import os, sys

# Setup environment for inference
os.environ["RANK"] = "0"
os.environ["WORLD_SIZE"] = "1"
os.environ["MASTER_ADDR"] = "127.0.0.1"
os.environ["MASTER_PORT"] = "29500"

# Assuming the LimiX repository is cloned and in the python path
# from inference.predictor import LimiXPredictor

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42)

# Download model weights
model_file = hf_hub_download(repo_id="stableai-org/LimiX-2M", filename="LimiX-2M.ckpt", local_dir="./cache")

# Initialize predictor (Requires local inference code from GitHub)
# clf = LimiXPredictor(device='cuda', model_path=model_file, inference_config='config/cls_default_retrieval.json')
# prediction = clf.predict(X_train, y_train, X_test)

# print("roc_auc_score:", roc_auc_score(y_test, prediction[:, 1]))
# print("accuracy_score:", accuracy_score(y_test, np.argmax(prediction, axis=1)))
```

# 3. Model Architecture & Pretraining

LimiX adopts a 12-block transformer architecture with axis-wise attention to features and samples. LimiX-2M specifically utilizes the **RaBEL** framework, which expands each scalar into compact localized RBF features to improve conditioning and shallow-layer effective rank. 

The model is pretrained through **Context-Conditional Masked Modeling (CCMM)**. By masking table cells and conditioning predictions on context rows, the model learns a wide range of conditional dependencies, allowing it to adapt to new datasets without task-specific training.

# 4. Deployment

For manual deployment, install the following dependencies:

```bash
pip install python==3.12.7 torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1
pip install scikit-learn einops huggingface-hub matplotlib networkx numpy pandas scipy tqdm typing_extensions xgboost kditransform hyperopt
```

*Note: Flash Attention 2 is recommended for optimal performance.*

# 5. License

1. **Code License**: The repository code is licensed under the [Apache-2.0 License](LICENSE.txt).
2. **Model Weight License**: The use of LimiX model weights is subject to a separate Model License:
   - Fully open for academic research.
   - Commercial use requires official authorization from [StableAI](https://www.stable-ai.ai/).

# 6. Citation

If you use LimiX-2M in your research, please cite the following:

```bibtex
@article{zhang2025limix,
  title={Limix: Unleashing structured-data modeling capability for generalist intelligence},
  author={Zhang, Xingxuan and Ren, Gang and Yu, Han and Yuan, Hao and Wang, Hui and Li, Jiansheng and Wu, Jiayun and Mo, Lang and Mao, Li and Hao, Mingchao and others},
  journal={arXiv preprint arXiv:2509.03505},
  year={2025}
}

@article{limix2m2026,
  title={LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models},
  author={Zhang, Xingxuan and others},
  journal={arXiv preprint arXiv:2606.04485},
  year={2026}
}
```