| --- |
| tags: |
| - pattern-classification |
| - multi-label-classification |
| datasets: |
| - maximuspowers/muat-fourier-5-medium |
| --- |
| |
| # Pattern Classifier |
|
|
| This model was trained to classify which patterns a subject model was trained on, based on neuron activation signatures. |
|
|
| ## Dataset |
|
|
| - **Training Dataset**: [maximuspowers/muat-fourier-5-medium](https://huggingface.co/datasets/maximuspowers/muat-fourier-5-medium) |
| - **Input Mode**: signature |
| - **Number of Patterns**: 14 |
|
|
| ## Patterns |
|
|
| The model predicts which of the following 14 patterns the subject model was trained to classify as positive: |
|
|
| 1. `palindrome` |
| 2. `sorted_ascending` |
| 3. `sorted_descending` |
| 4. `alternating` |
| 5. `contains_abc` |
| 6. `starts_with` |
| 7. `ends_with` |
| 8. `no_repeats` |
| 9. `has_majority` |
| 10. `increasing_pairs` |
| 11. `decreasing_pairs` |
| 12. `vowel_consonant` |
| 13. `first_last_match` |
| 14. `mountain_pattern` |
|
|
| ## Model Architecture |
|
|
| - **Signature Encoder**: [512, 256, 256, 128] |
| - **Activation**: relu |
| - **Dropout**: 0.2 |
| - **Batch Normalization**: True |
|
|
| ## Training Configuration |
|
|
| - **Optimizer**: adam |
| - **Learning Rate**: 0.001 |
| - **Batch Size**: 16 |
| - **Loss Function**: BCE with Logits (with pos_weight for training, unweighted for validation) |
| |
| ## Test Set Performance |
| |
| - **F1 Macro**: 0.2624 |
| - **F1 Micro**: 0.2737 |
| - **Hamming Accuracy**: 0.7323 |
| - **Exact Match Accuracy**: 0.0148 |
| - **BCE Loss**: 0.4683 |
| |
| ### Per-Pattern Performance (Test Set) |
| |
| | Pattern | Precision | Recall | F1 Score | |
| |---------|-----------|--------|----------| |
| | palindrome | 14.2% | 76.4% | 24.0% | |
| | sorted_ascending | 53.8% | 46.7% | 50.0% | |
| | sorted_descending | 13.4% | 96.2% | 23.5% | |
| | alternating | 18.6% | 80.7% | 30.2% | |
| | contains_abc | 17.8% | 84.6% | 29.4% | |
| | starts_with | 9.9% | 87.1% | 17.9% | |
| | ends_with | 39.7% | 78.9% | 52.8% | |
| | no_repeats | 13.7% | 63.0% | 22.4% | |
| | has_majority | 0.0% | 0.0% | 0.0% | |
| | increasing_pairs | 22.9% | 45.8% | 30.6% | |
| | decreasing_pairs | 14.3% | 92.9% | 24.8% | |
| | vowel_consonant | 0.0% | 0.0% | 0.0% | |
| | first_last_match | 26.7% | 45.1% | 33.5% | |
| | mountain_pattern | 16.8% | 89.6% | 28.3% | |
|
|
