EnigmaConsultant/huntr-poc-dl4j-hdf5-keras-import-oob-write

DL4J Hdf5Archive.readDataSet() native heap buffer overflow via HDF5 shape-bomb (Keras model import)

Gated PoC — access granted to maintainers/triagers on request.

Target: github.com/eclipse/deeplearning4j, package org.deeplearning4j.nn.modelimport.keras (Keras model import), file deeplearning4j-modelimport/src/main/java/org/deeplearning4j/nn/modelimport/keras/Hdf5Archive.java, method readDataSet(Group, String).

Summary

Hdf5Archive.readDataSet() computes the size of the Java array (and native buffer) it allocates to hold a weight tensor directly from the HDF5 dataset's declared dimensions, using plain 32-bit int arithmetic with no bounds or overflow checking:

case 1: /* Bias */
    dataBuffer = new float[(int) dims[0]];
    fp = new FloatPointer(dataBuffer);
    dataset.read(fp, dataType);   // <-- native HDF5 read, no size limit passed
    ...

(the 2D/3D/4D/5D cases multiply multiple long dims together and then cast to int the same way).

dims comes straight from DataSpace.getSimpleExtentDims() — i.e. straight from attacker-controlled HDF5 file metadata. An HDF5 file can declare a dataset with an enormous shape while storing zero actual data on disk (HDF5's chunked/incremental storage only allocates bytes for chunks that are actually written). If the declared element count, when truncated to a Java int ((int) dims[0], or (int)(dims[0]*dims[1]*...) for higher ranks), wraps around to a small positive number, the code allocates a tiny buffer — but then calls the native DataSet::read() with no memory-dataspace override, so the HDF5 C++ library writes however many elements the file dataspace actually declares (billions) into that tiny buffer: a classic native heap buffer overflow (CWE-787, Out-of-bounds Write).

This is reached directly from the public model-import API: KerasModelUtils calls weightsArchive.readDataSet(...) for every layer parameter while importing weights (KerasModelUtils.java ~lines 343-361), which is invoked from KerasModel/KerasSequentialModel during KerasModelImport.importKerasModelAndWeights(...) / importKerasSequentialModelAndWeights(...) — the standard, documented way to load a "Keras model" (.h5) file into DL4J. Any application that lets a user load/import an untrusted Keras .h5 model is directly exposed.

This is not the already-filed WordVector finding, and not CVE-2025-53001 / GHSA-wfhj-v5g7-vr7g (which is a separate bug: unsafe Java ObjectInputStream deserialization of the preprocessor.bin ZIP entry in ModelSerializer.java, in a completely different file/method). This finding is in the Keras-import HDF5 weight-reading path, a native-code memory-safety bug, not a Java deserialization bug.

Attacker input → sink

1. Attacker crafts a tiny (~1.4KB) .h5 file (bomb.h5 in this repo) containing one rank-1 float32 dataset foo with declared shape (4294968296,) = 2**32 + 1000 elements. Because nothing is ever written to the dataset, HDF5's lazy chunk allocation means the file stays tiny — see craft_bomb.py.
2. Victim application calls a standard DL4J Keras-import API (KerasModelImport.importKerasModelAndWeights(...) or, as isolated here for a minimal repro, Hdf5Archive.readDataSet("foo") directly — the exact same sink method).
3. (int) 4294968296L truncates to 1000 (low 32 bits) → new float[1000] (a 4000-byte buffer) is allocated.
4. dataset.read(fp, dataType) is called with no size limit; the native HDF5 library writes according to the real declared extent (~17.2GB worth of float32 elements) into that 4000-byte buffer.
5. Result: native heap buffer overflow → JVM segfaults (or, on a more favorable heap layout / with attacker-tuned offsets, potential memory corruption / RCE primitive — this PoC demonstrates the crash, which is sufficient to prove the out-of-bounds write; further exploitation to arbitrary code execution is a native-heap-grooming exercise beyond the scope of this PoC but is the well-understood next step for this bug class).

Real, reproducible evidence (not just static reasoning)

Built and run against the actual unmodified, currently-published Maven Central artifact org.deeplearning4j:deeplearning4j-modelimport:1.0.0-M2.1 (verified byte-for-byte identical Hdf5Archive.java source to the current master branch HEAD via diff against the -sources.jar, except for an unrelated openGroups() try/catch added later) plus the real org.bytedeco:hdf5 native JNI library (1.12.1-1.5.6, linux-x86_64).

Files in this repo:

• PoC.java — minimal harness that calls Hdf5Archive.readDataSet("foo") directly (the identical method/sink that KerasModelUtils calls internally for every layer weight during a real Keras import).
• pom.xml — Maven project pulling the real, unmodified released deeplearning4j-modelimport:1.0.0-M2.1 + org.bytedeco:hdf5 (native).
• craft_bomb.py — crafts bomb.h5 (the ~1.4KB malicious file) using h5py.
• bomb.h5 — the crafted malicious file (1400 bytes).
• control.h5 — negative-control file: a normal, valid 10-element dataset.
• hs_err_crash.log — the actual JVM fatal-error log produced when running java -cp <deps> poc.PoC bomb.h5.

Malicious file → crash

$ java -Xmx512m -cp <deps> poc.PoC bomb.h5
[*] Opening attacker-crafted HDF5 file: bomb.h5
[*] Calling Hdf5Archive.readDataSet("foo") - the exact sink used by
    KerasLayerUtils when importing Dense/Conv/etc. layer weights from
    a Keras .h5 model
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fb1ebb9f683, pid=1106436, tid=1106437
#
# Problematic frame:
# C  [libc.so.6+0xb1683]
...
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  org.bytedeco.hdf5.DataSet.read(...)V+0
j  org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.readDataSet(Lorg/bytedeco/hdf5/Group;Ljava/lang/String;)Lorg/nd4j/linalg/api/ndarray/INDArray;+875
j  org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.readDataSet(Ljava/lang/String;[Ljava/lang/String;)Lorg/nd4j/linalg/api/ndarray/INDArray;+17
j  poc.PoC.main([Ljava/lang/String;)V+50
...
siginfo: si_signo: 11 (SIGSEGV), si_code: 2 (SEGV_ACCERR), si_addr: 0x00007f9f64570f90

si_code: 2 (SEGV_ACCERR) = access to a validly-mapped-but-protected page — exactly what you'd expect from a write that runs off the end of a small heap allocation into a guard/protected page. Reproduced deterministically across 3 consecutive runs.

Negative control (proves it's the crafted shape, not environment/setup)

Running the exact same code path (Hdf5Archive.readDataSet) against control.h5 (a normal, valid 10-element float32 dataset with real data) does not crash — the native read succeeds fine, and the program only fails afterwards, at the unrelated Nd4j.create(...) call, because this minimal classpath deliberately omits an ND4J backend jar (irrelevant to the vulnerability, kept out only to keep the PoC's dependency footprint small):

$ java -Xmx512m -cp <deps> poc.PoC control.h5
[*] Opening attacker-crafted HDF5 file: control.h5
[*] Calling Hdf5Archive.readDataSet("foo") - the exact sink used by ...
Exception in thread "main" java.lang.ExceptionInInitializerError
    at org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.readDataSet(Hdf5Archive.java:295)
    at org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.readDataSet(Hdf5Archive.java:107)
    at poc.PoC.main(PoC.java:35)
Caused by: java.lang.RuntimeException: org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException: ...

This isolates the crash precisely to the oversized/overflowing declared shape in bomb.h5, not to any setup/environment issue: the only difference between the two runs is the dataset's declared extent, and only the malicious one crashes, at exactly the native read() call.

Reproduction

# 1. craft the malicious file
python3 -m venv venv && venv/bin/pip install h5py
venv/bin/python3 craft_bomb.py     # writes bomb.h5 (~1.4KB)

# 2. build/fetch the real, unmodified deeplearning4j-modelimport + hdf5 native jars
mvn dependency:build-classpath -Dmdep.outputFile=cp.txt

# 3. compile the PoC (no DL4J source modified — pure external caller)
javac -cp "$(cat cp.txt)" -d out PoC.java

# 4. run against the malicious file -> SIGSEGV
java -Xmx512m -cp "$(cat cp.txt):out" poc.PoC bomb.h5

# negative control (no crash, valid small dataset):
java -Xmx512m -cp "$(cat cp.txt):out" poc.PoC control.h5

Impact

• Denial of Service: guaranteed, trivial, deterministic JVM crash from a 1.4KB attacker-supplied "model" file, in any application that imports untrusted Keras .h5 models via DL4J (a documented, intended feature of the library).
• Memory corruption / potential RCE: the root cause is a genuine out-of-bounds native write of attacker-influenced size and offset past a heap allocation (the number of bytes written is directly the attacker's choice, via the HDF5 dataset shape metadata, and the destination buffer size is also attacker-influenced via the same integer-truncation trick). This PoC demonstrates the crash (sufficient to prove the primitive); turning it into controlled code execution would require the standard additional heap-grooming/exploitation work for this bug class, which is out of scope for a bug-bounty PoC but is the well-known next step for CWE-787 heap overflows of this shape.

Fix suggestion

• Validate dims[i] against sane bounds (and against Integer.MAX_VALUE for products) before allocating buffers, and reject/throw on out-of-range shapes (mirroring the MAX_BUFFER_SIZE_BYTES guard that readAttributeAsJson already has, but applied to readDataSet).
• Pass an explicit, size-matched memory DataSpace to DataSet.read(...) instead of relying on the default file-dataspace-sized read, so the native library can never write more than the destination buffer holds regardless of what the file claims.

Dedup / prior-art check performed

• Confirmed via gh/GitHub Security Advisories that eclipse/deeplearning4j has exactly two published advisories: GHSA-rc39-g977-687w (unrelated S3 bucket issue) and GHSA-wfhj-v5g7-vr7g / CVE-2025-53001 (unsafe ObjectInputStream deserialization of preprocessor.bin in ModelSerializer.java — a different file, different method, different bug class: Java object deserialization, not a native buffer overflow).
• Web-searched for prior reports of Hdf5Archive/readDataSet HDF5 shape-overflow issues in DL4J specifically — none found. The closest public analogue is CVE-2026-0897, a Python Keras (not DL4J) HDF5-shape-bomb DoS-only (OOM) bug in KerasFileEditor — a different codebase/language and a strictly less severe bug class (no int-truncation buffer overflow there, just huge legitimate allocation attempts).
• Confirmed the vulnerable Hdf5Archive.java source is unchanged between the current eclipse/deeplearning4j master HEAD and the latest published release (1.0.0-M2.1) via direct diff.