ROOT-TTree

Demonstrating the Coordinate Approach on ROOT TTree I/O

Author
Affiliation

SSCCS Foundation

SSCCS Foundation

Published

August 5, 2026

Abstract

This development plan describes how SSCCS will demonstrate the coordinate approach on the ROOT TTree I/O bottleneck: the same analysis, the same TTree API, and the same build chain, with the read path served from a coordinate-indexed store instead of the medium. ROOT’s TFile disk handling is never modified. The plan covers the approach, what the demonstration delivers, how it works, milestones, and boundaries.

Status: Draft development plan. The demonstration proceeds only after the engagement path in the CERN-SSCCS Collaboration Framework is accepted.

The Approach: The Same Stack, a Different Byte Source

The demonstration does not modify ROOT’s disk handling: TFile’s buffering, compression, and serialization remain exactly as they are. For byte ranges mapped into the Chton coordinate store, data is served from the store before any disk access happens; for everything else, the call proceeds through TFile unchanged. The existing stack runs as-is: the same executable, the same TTree API, the same build chain.

The byte-source boundary sits below TFile, at the POSIX I/O entry points:

researcher code (unchanged)
    ↓
TTree::GetEntry(entry)
    ↓
TBranch::GetEntry(entry)
    ↓
TTreeCache::GetEntry()
    ↓
TFile::ReadBuffer()
    ↓
[byte-source boundary]       ← standard dynamic-link interposition
    ├─ range mapped in the coordinate store → served from the store
    └─ otherwise → passed through to read()/pread()/mmap() unchanged

The working analysis treats TFile::ReadBuffer() as the symptom, not the cause, and points to the upper layers: a single event requests 153 data products spread across 8,190 branches; multithreaded runs calling GetEntry out of order can invalidate TTreeCache and collapse vector reads into singular reads; beyond 100,000 clusters, TTreeCache degrades from 7 seconds to over an hour. The documented result is 372,000 requests averaging 4.6 KB, an effective 33 KB/s, and a 14-hour workload. M1 confirms which term dominates.

The 14 Hours Appear to Be Overhead, Not Transfer

At raw bandwidth, 1.6 GB moves in seconds. The working hypothesis is that the 14 hours are per-request overhead: each of the 372,000 reads carries a system call, a page cache lookup, and often a context switch; the 8,190 fully split branches scatter event data across memory, collapsing cache locality; and 372,000 distinct small chunks keep the kernel page cache thrashing. M1 measures this hypothesis directly: the baseline trace separates system call, page cache, copy, and ROOT-internal costs, so the demonstration targets the measured dominant term. A first-order cost model of these terms accounts for roughly an hour of the fourteen; where the remaining time is spent is not yet attributed, and no projection is made until the baseline trace supplies the split.

The coordinate approach addresses this structurally: for mapped data, the read path never reaches the medium. The magnitude of the effect is a measurement, not a claim; the benchmark milestone reports the comparison against the ROOT baseline.

The removal is structural rather than incremental:

  • No per-access system call: a coordinate resolves in register arithmetic, and the store is reached through a mapped region, so no read() is issued per request
  • Cache locality: records are laid out contiguously in the coordinate space, giving the processor sequential, predictable access instead of 4.6 KB fragments
  • Page cache discipline: the store is demand-paged once with sequential locality, instead of 372,000 scattered chunks continuously evicting each other
  • Zero copy: mmap exposes the bytes directly, with no kernel-to-user copy per read

What the Demonstration Delivers

The demonstration is a single shared library, libtagma_cern.so, built with standard C++17 and CMake. Researchers run their analysis unchanged:

root -b -q analyze.C

With one environment variable added:

LD_PRELOAD=./libtagma_cern.so root -b -q analyze.C

The demonstration preserves:

  • The TTree API and existing analysis scripts, unchanged
  • ROOT’s disk handling: TFile’s buffering, compression, and serialization remain exactly as they are
  • The researcher’s build chain: g++ and CMake, with no new toolchain

It consists of three parts:

  1. A preparation tool (C++17, prebuilt binary) that converts the demo dataset (CMS Open Data 2016, MiniAOD) into a coordinate-indexed store
  2. The shared library that serves mapped data from the store and passes everything else through to ROOT
  3. A benchmark harness that runs the same logical workload against the ROOT baseline and reports wall-clock time, request count, and bytes moved

How It Works

The coordinate engine is closed-form arithmetic: composition and decomposition through \(C(i,m,f) = \text{U+AC00} + 588i + 28m + f\) and its inverse, with no hash function involved. An event is addressed as a coordinate, (run, luminosity block, event number), which resolves to a direct offset in the store.

The library is loaded through the standard dynamic linker (LD_PRELOAD) and decides at the POSIX I/O boundary (read, pread, mmap) whether a byte range is served from the coordinate store or passed through to ROOT unchanged, the same mechanism Darshan uses to trace ROOT I/O in production.

The demonstration deliberately stays at the byte-source boundary. Reaching into ROOT’s branch and cache traversal would require a ROOT-side extension and would change the deployment surface; it is out of scope. The result is the principle stated at the top: the same ABI, a different byte source; the researcher’s executable, the TTree API, and the build chain remain identical, and only the interior implementation changes.

Milestones

  1. M1: Reproduce the baseline: confirm the 372,000 x 4.6 KB singular-read signature from the 2025 Fermilab CCESOP analysis with a Darshan-style trace
  2. M2: Build the coordinate-indexed store from CMS Open Data 2016 (MiniAOD)
  3. M3: Build the shared library and validate the pass-through path against ROOT
  4. M4: Benchmark the Re-miniAOD pattern against the ROOT baseline: wall-clock time, request count, bytes moved, system call count, and cache miss rate
  5. M5: Release the demonstration, the benchmark harness, and the measured comparison as open source

Boundaries

  • The first demonstration covers fixed-width event records; variable-length collections (jets, tracks) are a later phase
  • Compression remains necessary at the PB scale; the demonstration uses selective or block-level compression
  • The demonstration complements ROOT and RNTuple; it does not replace them
  • All results are measured and reported as they are, with no projection ahead of measurement

After the Demonstration

If the measured comparison holds, the results provide the basis for discussing wider adoption and a potential contribution to the ROOT ecosystem. Everything in the demonstration is released as open source.