Getting started

Installation

TreeArrays is not registered. Add it from GitHub:

using Pkg
Pkg.add(url = "https://github.com/nsiccha/TreeArrays.jl")

Its hard dependencies are Statistics, Tables, and the Serialization standard library. NaNStatistics is a weak dependency that lights up nanquantile when you load it.

DynamicObjects is optional: loading it enables direct mmap caching of named results, including ragged trees and streamed statistics.

The mental model

A TreeData is a parent (the backing storage) plus dims (the named axes describing it). That is the whole type. What parent is decides which of four shapes you have — and these are dispatch-only aliases, so you always construct with TreeData(...):

Alias	parent is…	Meaning
TreeArray	an AbstractArray	a dense numeric leaf
TreeNamedTuple	a NamedTuple	a record — named fields form an axis
TreeTuple	a Tuple	a positional record
TreeRaggedArray	an AbstractArray{<:TreeData}	a ragged nesting of sub-trees

A TreeData is deliberately not an AbstractArray subtype. Only the array-backed leaf forwards size/getindex/iterate to its parent. See Feeding a numeric-array API for how to cross that boundary when a function insists on an array.

Naming your axes

using TreeArrays, Random
Random.seed!(1)

X = TreeData(randn(500, 3, 5), :draw, :subject, :time => [0.0, 1.0, 2.0, 4.0, 8.0])
TreeArrays.dims(X)

(draw: missing, subject: missing, time: [0.0, 1.0, 2.0, 4.0, 8.0])

The positional arguments after the backing array name its axes, in order. Three things are happening there:

• :draw and :subject are unlabelled axes — real dimensions with no coordinates attached. Their values is missing.

• :time => [...] is a labelled axis: the coordinates travel with the data.

• Keyword arguments become fixed coordinates — a single position, not an axis:

D = TreeData(randn(500, 3), :draw, :subject; dose = 20, arm = :treatment)
TreeArrays.dims(D)

(draw: missing, subject: missing, dose: 20, arm: :treatment)

dose = 20 is stored once, as a label. This is the constant-column smear (df[!, :dose] .= 20) that TreeArrays exists to retire — it costs O(1), not O(rows).

TreeData(:name) with no backing array throws

A TreeData always starts from its backing. TreeData(:time) alone throws and says so (before 0.2.0 it returned a builder closure: a constructor call that silently gave back a non-tree). To build one tree per item, as Ragged data does, map an explicit closure: map((v, t) -> TreeData(v, :time => t), values, times).

Reading the labels back

coords is the supported way to get an axis's coordinates:

coords(X, :time)

5-element Vector{Float64}:
 0.0
 1.0
 2.0
 4.0
 8.0

Labels come back exactly as provided — a Tuple stays a Tuple, a Vector stays a Vector, a range stays a range. The same convention holds for quantile's levels and selectdim's.

Do not reach for collect(dim)

A TreeDim has iterate/length, so collect runs and looks like the answer. It isn't: the eltype is Any, and for the two dim kinds that have no coordinates it invents one — Any[missing] for an unlabelled axis, Any[nothing] for a reduced ghost. coords errors by name for both, and the message names the spelling that would have worked.

Your first reduction

Every reduction names the axes it collapses and leaves the rest alone:

using Statistics
m = mean(X; dims = :draw)
TreeArrays.dims(m)

(subject: missing, time: [0.0, 1.0, 2.0, 4.0, 8.0], draw: nothing)

The :draw axis is not dropped — it is kept as an aggregated ghost (values === nothing), so provenance survives the reduction and the result still knows what it is. :subject and :time are untouched, labels included.

dims = nothing (the default) reduces the whole tree to a scalar:

mean(X; dims = nothing)

0.011955878763989162

Arithmetic keeps the labels

Broadcasting over a tree gives back a tree with the same dims:

centred = X .- mean(X; dims = :draw)   # the mean's reduced `:draw` stretches along X's
TreeArrays.dims(centred)

(draw: missing, subject: missing, time: [0.0, 1.0, 2.0, 4.0, 8.0])

Scalars combine with any tree. Two trees combine when they carry the same axes, by name and in the same order, with the same labels. An axis that one of them has reduced (like the mean's :draw above) or fixed stretches along the other's. Anything else refuses by name, and so does a plain array, whose axes have no names to match: wrap it as a tree, or pass it whole with Ref(x). Records broadcast field by field, ragged trees leaf by leaf.

Sizes read by name, too:

size(X, :draw), size(X, :time)

(500, 5)

Names are checked — a typo throws

try
    mean(X; dims = :drwa)
catch err
    println(err)
end

ErrorException("TreeArrays: dims=(:drwa,) names a dim that exists nowhere in this tree: (:drwa,). Available dims: (:draw, :subject, :time). `dims=` is foundALL -- a name that resolves nowhere is a typo, not an empty reduction.")

dims = is foundALL: every name must resolve somewhere in the tree. This is checked from the type, so a correct call costs nothing at runtime; only on a jagged tree does it fall back to an instance walk rather than let a typo through.

The rule behind this shows up everywhere in TreeArrays: never silently return a plausible-looking value. An earlier design returned missing for a dim that resolved nowhere, and it was retired precisely because a typo was then indistinguishable from a legitimately absent axis.

Small anatomy of the result

The whole tree renders itself, dims table included:

Y = TreeData(reshape(1.0:12.0, 4, 3), :draw, :time => [0.0, 1.0, 2.0])

TreeArray

dim	kind	coords
draw	axis	missing
time	axis	[0.0, 1.0, 2.0]

4×3 reshape(::StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, 4, 3) with eltype Float64:
 1.0  5.0   9.0
 2.0  6.0  10.0
 3.0  7.0  11.0
 4.0  8.0  12.0

mean(Y; dims = :draw)

TreeArray

dim	kind	coords
time	axis	[0.0, 1.0, 2.0]
draw	ghost	nothing

3-element Vector{Float64}:
  2.5
  6.5
 10.5

Display is always bounded and every elision is marked — string(td) is a preview, never a serialization. If you want the data, use Tables.columns or parent(td).

Where to go next

• Reductions — mapslices, quantile, coords = true, @kernel, chaining.

• Ragged data — per-subject series of differing length.

• Tables & plotting — melt long, pivot wide, feed a plotting layer.

• Comparisons — how this relates to DimensionalData.jl and FlexiChains.jl.