TreeArrays.jl

Metadata lives once, as an axis

A dim label is stored once and reused, never smeared across N rows. The fat pre-reduction frame that a groupby/combine/vcat pipeline builds simply never exists.

Ragged is first class

Per-subject series of differing length are a tree of sub-arrays over shared backing storage. The same reduction code runs on rectangular and ragged input.

Lazy assembly, eager compute

The big combined matrix is described structurally and never allocated; reductions stream over it block by block. No deferred compute graph, no per-element closures.

A Tables.jl source, for free

Records melt wide, axes melt long, and the columns are lazy views — @allocated stays flat across 100× rows. Plot it without ever building a DataFrame.

What it is

TreeArrays.jl gives you named, hierarchically-structured, possibly-ragged arrays over flat backing storage, plus streaming reductions that compute summary statistics over named axes.

It is built for one job: postprocessing. You already have the draws — from a sampler, from an ODE solve, from a prediction sweep. TreeArrays names their axes, lets you reduce over those axes by name, and hands the result to a plotting or diagnostics layer without ever materializing the cross-product of everything.

julia

using TreeArrays

X = TreeData(draws, :draw, :subject, :time => times)   # name the axes

quantile(                                     # 3. reduce :draw   → a ribbon axis
  quantile(                                   # 2. reduce :subject → a percentile axis
    mapslices(maximum, X; dims = :time),      # 1. reduce :time    → one number per (draw, subject)
    TreeDim(:percentile, (0.05, 0.5, 0.95)); dims = :subject),
  TreeDim(:ribbon, (0.025, 0.5, 0.975)); dims = :draw)

Each step keeps every axis it did not reduce, keeps the labels, and stays a TreeData. Nothing is stacked, pivoted or vcated in between.

The one load-bearing invariant

Eager compute / lazy assembly — no eager restructuring, ever.

Arithmetic runs now, in tight type-stable loops, on each materialized block. What is lazy is forming the big combined matrix: the cross-product of categorical and virtual axes is described structurally, and a reduction materializes one block, runs the kernel, discards it, and moves on.

This is deliberately not a deferred-compute / BroadcastArray design. The performance enemies here are allocating the big combined matrix and poor per-block kernel quality — not the cost of the individual evaluations.

Status

Version 0.2.0, unregistered, and young. It is used in anger for PKPD postprocessing. The public surface is listed on the API page under "Public API (frozen for 1.0)" and held there by a test; a handful of shapes error by name rather than being supported. Where a limit exists, the error says so; the package tries hard never to return a plausible-looking wrong answer.

Install it straight from GitHub:

julia

using Pkg
Pkg.add(url = "https://github.com/nsiccha/TreeArrays.jl")

TreeArrays.jlNamed, hierarchical, possibly-ragged arrays