EnigmaConsultant/huntr-poc-arrow-ipc-union-typeid-oob

Arrow IPC Union type_ids buffer not validated on read → out-of-bounds child-array vector index (SIGSEGV) on element materialization

Target: Apache Arrow C++ / PyArrow — Arrow IPC (Feather V2 / .arrow) reader Component: cpp/src/arrow/ipc/reader.cc — ArrayLoader::Visit(const UnionType&) Verified against: PyPI pyarrow==25.0.0 (libarrow.so.2500), Linux x86-64 Class: CWE-125 out-of-bounds read → CWE-787-adjacent wild pointer deref → SIGSEGV / DoS (untrusted IPC/Feather file)

---

Summary

The Arrow IPC reader loads a Union column's per-slot type_ids buffer (buffers[1]) — and, for dense unions, the offsets buffer (buffers[2]) — straight from the file with no validation of the content of the type_ids buffer against the union's declared type codes. A type_ids value that is a legal int8 but is not a declared union member survives:

• the default read path pyarrow.ipc.open_file(...).read_all() (read succeeds, rows=5), and
• the cheap Table.validate() that a careful caller runs (returns PASS).

Only validate(full=True) catches it — but ordinary consumption (to_pylist() / Array.GetScalar() / to_pandas()) dereferences the bad id first and crashes. On materialization the union scalar path maps each slot's type_id through the fixed 256-entry child_ids_ table; for an unregistered id child_ids_[type_id] == kInvalidChildId (-1), and the code then indexes child_data[-1] — an out-of-bounds std::vector<ArrayData> access — producing a wild ArrayData whose IsValid(offset) dereference faults.

Root cause

cpp/src/arrow/ipc/reader.cc, ArrayLoader::Visit(const UnionType&) (in the shipped libarrow.so.2500, the corresponding source lines ~480–508):

Status Visit(const UnionType& type) {
  int n_buffers = type.mode() == UnionMode::SPARSE ? 2 : 3;
  out_->buffers.resize(n_buffers);
  ...
  RETURN_NOT_OK(LoadCommon(type.id(), /*allow_validity_bitmap=*/false));
  out_->null_count = 0;

  if (out_->length > 0) {
    RETURN_NOT_OK(GetBuffer(buffer_index_, &out_->buffers[1]));       // type_ids: copied raw, content never scanned
    if (type.mode() == UnionMode::DENSE) {
      RETURN_NOT_OK(GetBuffer(buffer_index_ + 1, &out_->buffers[2])); // dense offsets: copied raw
    }
  }
  buffer_index_ += n_buffers - 1;
  return LoadChildren(type.fields());
}

GetFieldMetadata only copies node->length / node->null_count; there is no scan of the type_ids values against type.type_codes(). So a type_ids byte whose value is not a declared member id is admitted verbatim.

On element access (Array::GetScalar → arrow::internal::ScalarFromArraySlotImpl::Finish → Array::IsValid → IsNullSparseUnion / IsNullDenseUnion), the slot's type_id is mapped through the fixed 256-entry UnionType::child_ids_ table. For an id that is a valid int8 but not a declared member, child_ids_[type_id] == kInvalidChildId (-1). The code then does the equivalent of child_data[-1] (out-of-bounds std::vector<ArrayData> index), yielding a wild ArrayData pointer whose IsValid(offset) dereference segfaults.

Proof of concept

Build a 1-column union table with children int32 (type id 0) and string (type id 1) and a type_ids buffer [0,1,0,1,0]; write it with pyarrow.ipc.new_file. Locate the 5-byte type_ids buffer (bytes 00 01 00 01 00) in the file and overwrite one byte with an unregistered id (99 for sparse, 7 for dense) — a single-byte edit. Reload with pyarrow.ipc.open_file(...).read_all(): the read succeeds (rows=5) and Table.validate() PASSES. Materializing the column with .to_pylist() (i.e. Array::GetScalar) then segfaults.

Negative control = the unedited file, which reads, cheap-validates, full-validates, and materializes to [10, 'bb', 30, 'dd', 50] cleanly (exit 0).

Repro script union_typeid_repro.py and 4 artifact files are included: union_sparse_ok.arrow, union_sparse_badtypeid.arrow, union_dense_ok.arrow, union_dense_badtypeid.arrow.

python union_typeid_repro.py build     # write control + sparse/dense POCs
python union_typeid_repro.py control   # reads + materializes cleanly, exit 0
python union_typeid_repro.py sparse    # SIGSEGV (exit 139)
python union_typeid_repro.py dense     # SIGSEGV (exit 139)

Captured evidence (verbatim)

$ python union_typeid_repro.py control
read_all() OK rows=5
cheap Table.validate(): PASS  <-- validation gap
control: [10, 'bb', 30, 'dd', 50]
control exit=0

$ python union_typeid_repro.py sparse
read_all() OK rows=5
cheap Table.validate(): PASS  <-- validation gap
sparse exit=139

$ python union_typeid_repro.py dense
read_all() OK rows=5
cheap Table.validate(): PASS  <-- validation gap
dense exit=139

pyarrow 25.0.0 ; libarrow.so.2500

gdb backtrace (sparse)

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x00007ffff26fbadc in arrow::ArrayData::IsValid(long) const () from .../pyarrow/libarrow.so.2500
#1  arrow::internal::IsNullSparseUnion(arrow::ArrayData const&, long) ()
#2  arrow::Array::IsValid(long) const ()
#3  arrow::internal::ScalarFromArraySlotImpl::Finish() && [clone .constprop.0] ()
#4  arrow::Array::GetScalar(long) const ()
#5  __pyx_f_7pyarrow_3lib_5Array_getitem(...) ()  # -> Array.to_pylist
rip = arrow::ArrayData::IsValid(long) const+12

The dense variant is identical except frame #1 = arrow::internal::IsNullDenseUnion.

Impact

Any application that reads an untrusted Arrow IPC / Feather V2 file with a Union column and materializes it (to_pylist, to_pandas, GetScalar, scalar iteration) crashes with a SIGSEGV. The default read path and the routinely-used cheap Table.validate() both report success, so callers that "validate before use" are not protected. This is a remotely-triggerable denial of service via a crafted data file; the fault is an out-of-bounds std::vector index producing a wild-pointer dereference.

Suggested fix

In ArrayLoader::Visit(const UnionType&) (or a dedicated post-load check on the read path), scan the loaded type_ids buffer and reject any value not present in UnionType::type_codes() before the array is handed to consumers — i.e. fold the full-validation type_id range/membership check into the cheap validation performed on IPC read. The dense offsets buffer should likewise be bounds-checked against each child's length.

Dedup / prior-art note

This is distinct from the other Arrow IPC read-path findings in this series (compression -1 sentinel OOB, list offset OOB heap read, dictionary negative-index OOB, Feather string-offset OOB): the fault surface here is the Union type_ids content, unvalidated on read, reached through the union scalar IsNull{Sparse,Dense}Union → child_ids_[type_id] == -1 → children_[-1] path. No CVE was found covering unvalidated Union type_ids in the Arrow IPC reader as of the verification date (2026-07-16). validate(full=True) does detect it, confirming the underlying invariant exists but is not enforced on the default read/cheap-validate path.