proc.parity

The machinery behind the procgen rewrite's parity harness (procgenParity), its sheet tests (procgenSheet) and the procgen.coverage tool.

• parity.json: entries holds one entry per current op. Each names its new composition (new, node ids as nodebox.name, or nodebox.* for any node of a nodebox), its class (exact, meaning with an invariant, gone for a named decision with a note, unmapped where the sheet names no counterpart), and tolerance the comparison reads. An entry that states decided (the text of the ruling that accepts the difference between the op and its node) keeps running: the value differences its comparison finds are reported as accepted, under the verdict decided, and are held to nothing, while a node that raises and the cost findings stay findings (accepting, decidedLines). budgets states, per op, the read-backs its node may make ({ "<op>": { "readbacks": n } }); an op without one may make as many as the current op does. types maps a socket's type to the fixture a case builds for it. tolerance holds the global bounds (attribute, bounds, picture, costRatio, costFloorMs). suites names, per parity suite, the nodeboxes whose entries it compares (suites()); onNodes names, per on-nodes suite, the existing suites it runs with their ops answered by their nodes, and onNodesLeft the suites no on-nodes suite runs. cases holds a case per op that needs more than the defaults and names only content the library ships. A parity suite states the cases whose inputs are its test fixtures in the cases.json of its folder ({ "cases": { "<op>": case } }), since the library ships its test content nowhere; cases() is both together, one case per op, and an engine whose library leaves the suite out finds no case for those ops.
• A case states params, fixtures (a fixture by input socket), node (the new node a multi-node entry compares against), nodeBy ({ param, nodes = { value = node } }: for an entry whose op one node per value of a param answers, the node an existing suite's call reaches for each value, which standIns runs; a value it names no node for is refused), rename (inputs, params, outputs: the new node's names where they differ, false leaves a name out), compare (bounds: a Geometry is held to its bounds alone, for an output whose element layout differs by design; ramp: two ramps are held to the values they read at the same positions), sink (asset or handles: the op writes to the world, each side under its own name and what it wrote is compared and removed), world (entities spawned for the run; scene takes the run's root scene, settle waits until the op runs on them), tolerance (bounds that replace the global ones for this case, { attribute = 0.05 }: for an op whose values come from the device's sampler, whose fixed-point weights and decode answer within a fraction of a percent of the exact blend a node computes), ignore (outputs left out), unreachable (the sentence that says why the harness has no fixture for the case; the entry then counts as a harness error) and, for an op that is a solver over a timeline, suite and describe (the group of an existing suite that holds its comparison; deferred leaves that group to the suite's own run and states the entry pending, for a group that costs the suite's whole time a second time inside this one).
• A fixture is a name (mesh, points, curve, positionField, or a type named in types), a current op run on its own fixtures ({ op, params, inputs, output }; the new side builds it through the node that replaces the op when the entry for that op names exactly one, so a ramp, a group, a prefab, a raster or an Sdf a node reads is one that library made; shared: true makes both sides take what the current op makes, for inputs the entry under test is held to and the op that made them is not), a node run on its own fixtures ({ node, params, inputs, output }, a new node by id), a list ({ list }), a record ({ record }) or a literal ({ value }). A case's newInputs are fixtures only the new side takes, for an input the current op states as a param (a texture the node reads where the op names the asset).
• sheet.json: the composition sheet's expectations: the eight ops, the 23 kernel families with their tallies (a kernel asset <kernel>_<pass> belongs to the family <kernel>), the layers, the L2 ids as the core package names them, universalUses (the L2 nodes every row may call), one row per node family (the kernel families, L2 nodes and ops a node of the row may name; authorKernel for rows whose nodes run a kernel the author names; which nodes may stand on one kernel family alone), and the reported numbers.

scan() finds the new parts in the library: the nodes of every built-in nodebox wherever its package keeps it, the kernel families with their pass assets, and the modules that sit beside nodes. analyze(src, ...) reads a node's source: the nodes it calls (g.<name>(...)), the ops it names (op.<name>), the kernel families it names, a structural hash per call, and WGSL text in its strings. rowViolations(scan, node), dispatchViolations(scan, node) and directCallers(scan) state what a node breaks of the sheet. valuesHold(result) reads a comparison's values alone, which is what lets an existing suite run on the nodes of an op whose cost is over. assess(scan, entry) states where an entry stands (decided, unmapped, pending, ready); compare(scan, entry) runs a ready entry's current op beside its node; checkCase(entry) runs the current op on the entry's case. redirect(map, fn) runs fn with registry.get answering a new node for each op id in map; runSuite(name, redirect) runs an existing suite's cases through the test framework's own runner, and runSuiteOnNodes(scan, name, ops) runs it as it stands and with its ops answered by their new nodes. report(opts) is the coverage report.

A comparison states cost as separate numbers (result.cost): oldMs (the current op's CPU time), planMs (the evaluation outside the nodes' own cooks: expanding the call form, lowering refs and constructs, scheduling, with the plan's share of the waiting taken out), cookMs (the CPU time the nodes' cooks spend, with their share of the waiting taken out), waitMs (the time in task.wait, a read-back's frames, shared between the cooks and the plan by the read-backs each made), dispatches (device dispatches queued), readbacks (read-backs the substrate counted) and frames (frames that passed during the evaluation). A Field in an output is read over the Geometry its side was given, a Column by its values, and a Bool answers 1 or 0. The cost ratio of tolerance is applied to cookMs against oldMs; the read-back count is asserted against the op's budget on its own, so a read-back removed from a node shows as frames falling to zero. Times are the median of three runs; counts are the last run's.

A comparison states its values and its cost apart. result.valueFindings are the differences in what the node answers and result.costFindings the cost and read-back findings (result.findings holds both). verdict(assessed, result) reads them as value (equal, differ, missing for an attribute, layer or column the new output lacks, raised for a node that failed on the case's inputs, harness for a case whose inputs could not be made, skipped, decided) and cost (within, over, none); summary(scan) counts every entry under exactly one of pass, costOver, differ, missing, raised, harness, skipped and decided and reports the same per family (the nodebox of the entry's first new node) with the median and worst cost ratio.

What is compared where the two libraries carry a value differently: a Geometry by its live counts (a domain allocated at a capacity compares its live front), a Raster layer by layer at the grid's cell centres, a Texture by size, channel count and texels (bytes scaled to 0 to 1 and held to half a byte step), a count the new library holds on the device as a one-word column by its value, a Population against the instanced points by instance count, and a list of numbers against a list of records of numbers as one layout finding that says whether the numbers agree.