---
license: mit
base_model: google/vit-base-patch16-224
library_name: transformers
pipeline_tag: image-classification
tags:
  - bwsk
  - combinator-analysis
  - vit
  - reversible-backprop
  - convergence-training
datasets:
  - cifar10
metrics:
  - accuracy
model-index:
  - name: bwsk-vit-base
    results:
      - task:
          type: image-classification
          name: Fine-tune (Conventional)
        dataset:
          name: cifar10
          type: cifar10
        metrics:
          - name: accuracy
            type: accuracy
            value: 0.9759
            verified: false
      - task:
          type: image-classification
          name: Fine-tune (BWSK Analyzed)
        dataset:
          name: cifar10
          type: cifar10
        metrics:
          - name: accuracy
            type: accuracy
            value: 0.9819
            verified: false
      - task:
          type: image-classification
          name: Fine-tune (BWSK Reversible)
        dataset:
          name: cifar10
          type: cifar10
        metrics:
          - name: accuracy
            type: accuracy
            value: 0.9734
            verified: false
      - task:
          type: image-classification
          name: From Scratch (Conventional)
        dataset:
          name: cifar10
          type: cifar10
        metrics:
          - name: accuracy
            type: accuracy
            value: 0.3747
            verified: false
      - task:
          type: image-classification
          name: From Scratch (BWSK Analyzed)
        dataset:
          name: cifar10
          type: cifar10
        metrics:
          - name: accuracy
            type: accuracy
            value: 0.3694
            verified: false
      - task:
          type: image-classification
          name: From Scratch (BWSK Reversible)
        dataset:
          name: cifar10
          type: cifar10
        metrics:
          - name: accuracy
            type: accuracy
            value: 0.3775
            verified: false
---

# BWSK ViT-base

**ViT-base** (86M params) trained in **6 variants** (3 BWSK modes x 2 experiments) on CIFAR-10 with full convergence training and early stopping.

This repo contains all model weights, configs, and training results in a single consolidated repository.

## What is BWSK?

BWSK is a framework that classifies every neural network operation as **S-type** (information-preserving, reversible, coordination-free) or **K-type** (information-erasing, synchronization point) using combinator logic. This classification enables reversible backpropagation through S-phases to save memory, and CALM-based parallelism analysis.

## Model Overview

| Property | Value |
|----------|-------|
| **Base Model** | [google/vit-base-patch16-224](https://huggingface.co/google/vit-base-patch16-224) |
| **Architecture** | Vit (image_cls) |
| **Parameters** | 86M |
| **Dataset** | CIFAR-10 |
| **Eval Metric** | Accuracy |

## S/K Classification

| Type | Ratio |
|------|-------|
| **S-type** (information-preserving) | 72.1% |
| **K-type** (information-erasing) | 27.9% |

## Fine-tune Results

| Mode | Final Loss | Val Accuracy | Test Accuracy | Peak Memory | Time | Epochs |
|------|------------|----------|----------|----------|----------|----------|
| Conventional | 0.0022 | 97.8% | 97.6% | 3.1 GB | 3.8m | 1 |
| BWSK Analyzed | 0.3425 | 98.0% | 98.2% | 3.1 GB | 8.4m | 2 |
| BWSK Reversible | 0.0019 | 97.7% | 97.3% | 2.0 GB | 4.5m | 1 |

**Memory savings (reversible vs conventional):** 37.3%

## From Scratch Results

| Mode | Final Loss | Val Accuracy | Test Accuracy | Peak Memory | Time | Epochs |
|------|------------|----------|----------|----------|----------|----------|
| Conventional | 1.5347 | 37.9% | 37.5% | 3.1 GB | 7.6m | 2 |
| BWSK Analyzed | 1.8406 | 38.0% | 36.9% | 3.1 GB | 4.3m | 1 |
| BWSK Reversible | 1.8934 | 39.6% | 37.8% | 2.0 GB | 6.4m | 2 |

**Memory savings (reversible vs conventional):** 37.3%

## Repository Structure

```
├── README.md
├── results.json
├── finetune-conventional/
│   ├── model.safetensors
│   ├── config.json
│   └── training_results.json
├── finetune-bwsk-analyzed/
│   ├── model.safetensors
│   ├── config.json
│   └── training_results.json
├── finetune-bwsk-reversible/
│   ├── model.safetensors
│   ├── config.json
│   └── training_results.json
├── scratch-conventional/
│   ├── model.safetensors
│   ├── config.json
│   └── training_results.json
├── scratch-bwsk-analyzed/
│   ├── model.safetensors
│   ├── config.json
│   └── training_results.json
├── scratch-bwsk-reversible/
│   ├── model.safetensors
│   ├── config.json
│   └── training_results.json
```

## Usage

Load a specific variant:

```python
from transformers import AutoModelForImageClassification, AutoFeatureExtractor

# Load fine-tuned conventional variant
model = AutoModelForImageClassification.from_pretrained(
    "tzervas/bwsk-vit-base", subfolder="finetune-conventional"
)
```

## Training Configuration

| Setting | Value |
|---------|-------|
| **Optimizer** | AdamW |
| **LR (fine-tune)** | 5e-05 |
| **LR (from-scratch)** | 3e-04 |
| **LR Schedule** | Cosine with warmup |
| **Max Grad Norm** | 1.0 |
| **Mixed Precision** | AMP (float16) |
| **Early Stopping** | Patience 3 |
| **Batch Size** | 16 |

## Links

- [GitHub Repository](https://github.com/tzervas/ai-s-combinator)
- [Whitepaper](https://github.com/tzervas/ai-s-combinator/blob/main/docs/WHITEPAPER.md)
- [Full Training Report](https://github.com/tzervas/ai-s-combinator/blob/main/docs/FULL_TRAINING_REPORT.md)

## Citation

```bibtex
@software{zervas2026bwsk,
  author = {Zervas, Tyler},
  title = {BWSK: Combinator-Typed Neural Network Analysis},
  year = {2026},
  url = {https://github.com/tzervas/ai-s-combinator},
}
```

## License

MIT