# FirstRun
The card a newcomer meets: four steps that tick themselves off from the state
of the network and the fly, ending in the sentence the world is for.
```luau
entity.spawn("FirstRun").component.add("FirstRun", { brain = brainEnt, fly = flyEnt })
```
The steps are read in order, so the card cannot tick ahead of the person
reading it:
1. Reset the network. Ticks when the network's own clock drops.
2. Read out the proboscis motor neurons. Ticks when a group named MN9 exists.
3. Taste something sweet. Ticks when MN9 passes 50 Hz.
4. Look at the fly. Ticks when the fly's proboscis is out.
Then the line: "MN9 is firing at N Hz and the proboscis is out. You just drove
a real connectome.", and a button that opens the experiments panel.
The card drives nothing. It reads `describe()`, `rates()` and the fly's
`motor()`, and every step is something the visitor did at the bench.
Completion is stored as the attribute `fruitFly.firstRun` on this entity, so
the world remembers rather than one browser, and the card does not come back
for whoever finished it.
The value names who that was. An entity attribute is authored state: it is
serialized into the scene and published with it, and the entity is shared by
everyone in the world. A value naming somebody else, including one a published
scene shipped, leaves the card standing, so a visitor always meets a fresh
card whatever the last visitor did.
`resetOnAwake` forgets it at load, and `comp:restart()` forgets it now, which
is how the card is met a second time. `comp:progress()` reports which steps
have ticked.
The card takes the right column above where the experiments panel opens, laid
out from `ui.screenSize()`, and it yields to the panel: while the panel is
open the card is not drawn.
# FoodPatch
A disc of food on the ground. A `FruitFly` finds every patch in the scene and
drives its tarsal taste neurons by the patch's sweetness while a front foot is
inside the disc, and its labellar taste neurons while the extended proboscis
is. The disc draws itself, tinted by how sweet it is.
A patch carries sweetness and nothing else. MaleCNS v1.0 labels no bitter
gustatory receptor neuron, so there is no honest cell type to drive with a
bitter reading; whether such a path exists in the data is one of the things
the bench is for.
Fields: `radius`, `sugar`, `height`.
Methods: `contains(x, y, z)`.
```luau
local food = entity.spawn("Food")
food.position = { 2.5, 0, 1.5 }
food.component.add("FoodPatch", { radius = 0.6, sugar = 1.0 })
```
# FruitFly
An embodied fly driven by a `FruitFlyBrain`. Every frame it writes the world
into the network's sensory neurons and reads a motor command out of named
motor and descending cell types.
Which side of the body a sense or a command belongs to is the dataset's
answer, not this component's. MaleCNS v1.0 labels a side on the looming
detectors and on the leg motor neurons, and labels the tarsal and labellar
taste neurons "unknown", so looming is driven per eye and taste is not.
| sense | cell types | driven by |
|---|---|---|
| tarsal taste | `claw_tpGRN`, `dorsal_tpGRN` (60, side unknown) | either front foot on a `FoodPatch` |
| labellar taste | `LB*` (165, side unknown) | the extended proboscis on a `FoodPatch` |
| looming, left eye | `LPLC2` side L (94), `LC4` side L (71) | anything growing in the left eye's field, the arena wall included |
| looming, right eye | `LPLC2` side R (91), `LC4` side R (55) | the same for the right eye |
The two looming classes are separate drives, so silencing `LPLC2` leaves `LC4`
driven and the ablation has an answer.
| command | cell types |
|---|---|
| walk | leg motor neurons (`vnc_motor` types ending in ` MN`), left 189 and right 185, averaged |
| turn | `DNa01` + `DNa02` left minus right, plus the leg motor neurons right minus left |
| proboscis | `MN9` |
| flight | `DLMn` + `DVMn` wing power-muscle motor neurons |
| jump | a `DNp01` burst above `escapeHz` |
Beating wings lift the fly only once it is already off the ground: on six legs
the beat is a beat, and the legs keep walking underneath it.
## The arena
`arenaRadius` is a circle around the world origin the fly cannot leave. Inside
`arenaMargin` of it the wall is read by each eye as something looming, so the
fly meets the wall with the circuits it already has; the body is clamped at
the radius as well, so whatever the circuits do the fly is still in the arena.
The wall is only a stimulus while the fly is carrying itself into it, and the
detectors adapt to it over `WALL_ADAPT_S` and recover over `WALL_RECOVER_S`.
A fly walking at the edge gets the burst that turns it; a fly resting against
the edge is looking at something that has stopped arriving, and drives
nothing. That matters beyond the fly: a standing drive on 165 looming neurons
puts the whole network into a spreading state, and every reading anyone takes
in this world is taken from that network.
`escapeHz` is a RISE, not a level. The reading is how far `DNp01` has gone
above the rate it has been holding, averaged over `ESCAPE_BASELINE_S`. The
network's own recurrence holds the giant fibre at a standing rate with nothing
looming at all, and a threshold on the absolute rate reads that as a permanent
emergency and hops the fly for ever.
Fields: `brain`, `bodyLength`, `walkSpeed`, `turnRate`, `legTurnGain`,
`motorFullHz`, `flightFullHz`, `tasteHz`, `loomHz`, `loomFov`, `escapeHz`,
`arenaRadius`, `arenaMargin`, `groundY`, `wander`, `wanderTurn`.
`wander` and `wanderTurn` are the walking and the turning this fly does that
its connectome is not asking for, and `motor()` reports both apart from
`brainWalk` and `turn`, which are the network's. Set them to zero for a fly
that does only what the wiring says; leave them and a quiet network leaves a
fly that explores its arena instead of walking one straight line to the edge.
Methods: `motor()`, `senses()`, `readouts()`, `sensoryNeurons()`, `drives()`,
`soundVolumes()`. Events: `onJump`, `onFeed` (once per meal).
```luau
local fly = entity.spawn("Fly")
fly.component.add("FruitFly", { brain = entity.find("Brain"), bodyLength = 0.6, wander = 0.15, arenaRadius = 12 })
```
`motor()` reports `brainWalk` (the leg-motor term alone) and `wander` (the
baseline added to it) beside `speed`. `senses()` reports `sugarFootL` and
`sugarFootR` per front foot, `loomLPLC2` and `loomLC4` in Hz, `loomSide`
(-1 the left eye to +1 the right) and `wall` (0 to 1).
The fly's own drives are filed under the source `fly`, so clearing them leaves
the bench's and a protocol's standing.
On a shared world the entity's owner writes the senses into the brain (a
synced call); every peer poses its own copy from its own network.
# FruitFlyBrain
A whole connectome running as a spiking network on this entity. It loads the
`connectome` asset the field names (the package's `maleCNS` by default),
steps it every frame the gameplay clock runs and, with `runInEdit`, in edit
mode too; draws every neuron with a soma as a lit point around the entity;
plays the network's own crackle from it; and hands the live network to
anything that asks.
Fields: `connectome`, `running`, `runInEdit`, `timeScale`,
`maxStepsPerFrame`, `gain`, `minSynapses`, `showCloud`, `cloudGate`,
`cloudScale`, `neuronRadius`, `bloom`, `bloomIntensity`, `crackle`. The
defaults are the calibrated regime: gain 0.6, minSynapses 5, timeScale 1,
maxStepsPerFrame 32.
Methods on the ref: `getBrain()`, `stimulate(sel, hz, source?)`,
`silence(sel, on)`, `clearStimulation(source?)`, `drives()`, `reset()`,
`watch(name, sel)`, `unwatch(name)`, `rates()`, `classRates(limit?)`,
`groupActivity(name)`, `neuronAt(screenX, screenY, radiusPx?)`,
`keepCloudDrawn(seconds?)`, `describe()`. `stimulate`, `silence`,
`clearStimulation` and `reset` are synced calls, so every peer's copy of the
network receives the same inputs. `onLoaded` fires with the neuron and edge
counts once the network is on the GPU.
```luau
local e = entity.spawn("Brain")
e.position = { 0, 5, -8 }
e.component.add("FruitFlyBrain", { gain = 0.6 })
local c = e.component.get("FruitFlyBrain")
c:watch("MN9", { types = { "MN9" } })
c:stimulate({ types = { "claw_tpGRN" } }, 100, "bench")
```
## The visibility gate, and what it means for a capture
The cloud is 139,741 instances, and a registered instanced population has its
whole transform run recomposed on the CPU every frame whether or not the cull
draws any of it. Measured on this world: `render.mesh_transform_compose` is
4.74 ms a frame with the cloud registered and 0.67 ms with it down. So the
component tests the cloud's bounding sphere against the on-screen camera's
frustum every three frames, registers the draw the moment the sphere enters,
and drops it after thirty consecutive frames outside. `showCloud = false`
still takes everything down.
The gate only knows about the camera on screen. **An offscreen capture aimed
at the brain from a station the player is not standing at can therefore come
back with no cloud in it.** Call `keepCloudDrawn(seconds)` first, or set
`cloudGate` false.
Moving the entity moves the cloud with it: the transforms are rewritten from
the neuron records a slice a frame, with no rebuild.
## Drives are owned
`stimulate` takes the name of whoever is driving. The rate a neuron sees is
the largest any source asked for, so the fly stepping onto a food patch
(`"fly"`) cannot take away what the bench (`"bench"`) or a protocol
(`"protocol"`) is driving, and `clearStimulation("fly")` clears only the
fly's. A call that names no source is recorded as `"code"`. `drives()` is what
the bench lists.
## Sound and light
While the cloud is drawn the component registers the builtin bloom on the
`scene` layer, which is what turns a spiking neuron from a bright dot into a
point of light, and takes it off again when the cloud goes down. The Audio
component it puts on the entity plays the package's `crackle` clip on a loop
with its volume following the whole-network rate: silent below 0.05 Hz,
rising to 0.35 by 5 Hz.
# FruitFlyExperiments
The experiments panel: the five experiments from `experiments.guide` as one
press each, with a verdict line computed from the run's own numbers and a
compare overlay against the run before.
```luau
local panel = entity.spawn("Experiments")
panel.component.add("FruitFlyExperiments", { brain = brainEnt, fly = flyEnt })
```
`brain` is the entity carrying the `FruitFlyBrain`. `fly` is the embodied fly
the looming protocol is recorded against.
Pick a protocol, press RUN. Every run clears the drives, resets every
membrane, records `Protocol.BASELINE_MS` of the network before its first drive
starts, applies its own gain and ablations, samples every readout on the
network's own clock, and puts everything back. A protocol with variants runs
them in order: gain recovery is three runs, looming and ablation are two each.
COMPARE lays the run before the newest one over the same plot, dimmed, and
prints its verdict under the new one.
## Controls
| Action | Keyboard | Gamepad | Touch |
|---|---|---|---|
| Show or hide | P | Select | The "Experiments" button, bottom right |
| Next protocol | N | Right stick press | Tap the protocol |
| Previous protocol | B | None: the list wraps | Tap the protocol |
| Run | R | Right trigger | RUN |
| Compare | C | Left trigger | COMPARE |
The five protocols are the panel's focus ring, and the focused one carries a
2 px accent border. Moving the focus is choosing the protocol, so a pad
reaches every protocol without an activate button. A text field holding the
caret owns the keyboard, so a selection typed at the bench does not run a
protocol on its way in.
The panel lays out from `ui.screenSize()` every render: the right 40 percent
of the screen, from 7 percent down, clear of the bench's left column. The
window can be dragged and resized from there.
## From code
`comp:start(id)` runs a protocol by id (`taste`, `auditory`, `gain`,
`looming`, `ablation`) exactly as RUN does, and `comp:runs()` is the session's
runs, each carrying its label, its verdict, its samples and the statistics the
verdict was computed from. The protocols themselves and the machinery that
runs them are in `protocol.module`, which has no UI in it.
# FruitFlyLab
The bench for a `FruitFlyBrain`. A movable, resizable window, laid out from
`ui.screenSize()` so it reads the same on a phone held upright and on a wide
monitor, holding everything a person does to the network by hand.
From the top:
- **Header.** The connectome's name and its neuron and connection counts,
with thousands separators.
- **State.** Simulated seconds and the mean rate over the whole network on
one line, spikes and dropped spikes on the next, in a tabular face. When
the network is above 0.05 Hz with no drive live, a line says so: the
activity is recurrence in the wiring, and reset is the cure.
- **Run / pause, reset, clear drive**, then the three sliders (gain, time
scale, minimum synapses) in one fixed label column, so the tracks and the
value pills line up. The pill is the value; the label is the name alone.
- **Readouts.** One row per watched group: an `x` that stops watching it, the
group's name in its own colour, a bar drawn to that group's own full scale
with the scale printed beside it, and the rate. The rows scroll inside a
capped block, and groups sitting at zero collapse into one dimmed line with
a `show` that opens them. Below, the trace: every group on one plot over
the last 24 seconds, on a square root scale so a group at 4 Hz stays
visible beside one at 130, with both gridline values named in the section
header. A group of 32 neurons or fewer offers a raster, one row per neuron
with a tick wherever that neuron was active at the sample.
- **Hot classes.** The ten cell classes with the highest mean activity right
now, as bars. Activity is the model's own 60 ms spike trace, not a rate,
and the block says so.
- **Live drives.** What is being driven at this moment and by whom: the
bench, the fly, or a protocol.
- **Selection.** Eight chips for the cell types this bench is used on most,
a free text field, the rate slider, then stimulate / silence / restore, and
watch / count with a name field.
- **Neuron.** What the last identified neuron is, with a button that fills
the selection with its cell type.
- **Fly.** What the fly senses and what it does, as the fly reports them.
- **Legend.** A swatch per colour in the neuron cloud, with the dataset's own
superclass names beside it.
A group keeps its colour for the life of the session: the colour comes from a
hash of the group's own name into a twelve colour palette, so adding a group
never recolours the others.
## Selections
The grammar lives in `selection.module` beside this component, and its README
is the table of terms. `lab:parseSelection(text)` is the bench's way in to it
and returns the same `(selection, error)` pair it always did.
## Controls
`lab.inputMap` carries the bench's own controls on keyboard, gamepad and
touch: `labToggle` (L, Start, the Lab button), `labFocusNext` and
`labFocusPrev` (arrows or Tab, D-pad), `labActivate` (Enter, north),
`labAdjustLeft` and `labAdjustRight` (arrows, D-pad, 5 percent of the
slider's range) and `labIdentify` (mouse left, west, the ID button). The
focused control draws a 2 px accent border, so every control is reachable
with a pad alone or a keyboard alone. A "Lab" button sits at the bottom left
of the screen for a finger, and the window's own hide and show button is
where it always was.
`labIdentify` names the neuron under the pointer, or under the screen centre
when nothing has moved the pointer recently, which is what a pad and the
touch button aim with. A press the bench's own window took is ignored.
## Fields and methods
Fields: `brain`, `fly`, `open`, `refreshHz`, `inputMap`.
Method: `parseSelection(text)`.
```luau
entity.spawn("Lab").component.add("FruitFlyLab", { brain = entity.find("Brain"), fly = entity.find("Fly") })
```
## What the bench reads from the network
Everything the bench draws comes from the `FruitFlyBrain` on the `brain`
entity: `describe()` for the header, the state line and every group's size
and rate, `classRates()` for the hot classes, `drives()` for the live drives,
`groupActivity(name)` for a raster, and `neuronAt(x, y)` for a pick. A
network that publishes none of the last four still draws every other block,
and the blocks that need them say what is missing rather than showing a
number nobody computed.
The `experiments` guide (`guides { path = "experiments" }`) is the manual.
# FruitFlyLook
The lab's look, as data on one entity. Put it on an entity called `Look`,
point its `sun` and `ambient` Fields at the scene's two lights, and every
decision about how the room is lit and graded is in one inspector.
It owns four things:
- **The lighting intent.** `sunColor`, `sunIntensity`, `sunDirection`,
`ambientColor`, `ambientIntensity`. It writes these onto the two lights when
it wakes and whenever a Field changes, so the lights themselves are a cache
of what this component says.
- **The cast shadows.** `shadowSoftness`, `shadowFade`, `shadowDistance`, via
`renderer.setShadowConfig`. At softness 0 every shadow is a hard stencil;
the lab runs at 0.55.
- **The haze.** A depth fog registered as the post effect `fruitFlyHaze` on
the `scene` layer at priority 60, so it runs before anything else the world
adds and leaves every widget on screen as authored. `hazeDensity` is how
much air a metre of depth carries; `hazeFar` decides how much of the sky it
tints, because the sky reads as sitting at that plane. Setting
`hazeDensity` to 0 removes the effect.
- **The vignette.** Registered as `fruitFlyVignette` on the `scene` layer at
priority 200, so it lands last. `vignetteIntensity` 0 removes it.
## What it is for
The floor runs to a hard polygonal edge against the sky wherever it ends, and
a specimen suspended in the air reads as a cut-out against a clean gradient.
The haze answers both: the floor's far edge dissolves into a colour matched to
the sky near the horizon, and the air between the viewer and the specimen
separates it from the sky behind it. The vignette settles the eye on the
middle of the frame without taking anything out of it.
## Methods
| Call | What it answers |
|---|---|
| `look:apply()` | Restates every part of the look on the engine. The wake and the inspector do this for you; call it after something else has moved the lights, the shadow config or the post chain. |
| `look:describe()` | What the look is asking for right now, and the shadow config the engine answered with. |
## What it does not touch
Only the two effects it registers by name, the two lights its Fields name, and
the directional shadow config. Anything else the world puts in the post chain
runs beside it under its own name.
# arenaRing
A flat annulus in the XZ plane, inner radius 11.85, outer radius 12.15, 128
segments, normals up. It is the painted line on the lab floor marking the
12 metre arena the `FruitFly` component's `arenaRadius` keeps the fly inside.
Place it on an entity a little above the floor (`ArenaRing` sits at y = 0.02)
with a Model and a tint; it casts no shadow.
The radius is baked into the geometry rather than taken from the entity's
scale, so the ring cannot drift out of agreement with `arenaRadius` by
somebody resizing the entity. If `arenaRadius` ever moves, re-author the mesh.
# brain
A whole connectome running as a leaky integrate-and-fire network on the GPU.
`Brain.load` binds any `connectome` asset to the package's kernels and returns
a network; the `FruitFlyBrain` component is the usual owner, and anything
holding the network can step, drive, cut, read out and draw it.
## Exports
- `M.load(connectomeRef, params?) -> Brain`: allocate the network at rest.
`params` overrides any key of `Params`: `dtMs`, `vRest`, `vThresh`,
`vReset`, `tauM`, `tauSyn`, `refracMs`, `delayMs`, `weightMv`, `gain`,
`minSynapses`, `traceMs`, `maxSpikes`, `seed`, `timeScale`,
`maxStepsPerFrame`. A key that is not one of those raises, as `setParams`
does, and so does a value that is not a number.
- `brain:update(dt)`: once a frame: queue the steps `dt` covers, issue and
drain the counter readback, and fold the activity snapshot when something
has asked for one.
- `brain:step(n)`: queue `n` steps outright.
- `brain:select(sel) -> { index }`: the neurons a selection names, ascending.
The array is the network's own and memoized on the selection, so read it
rather than writing into it.
- `brain:stimulate(sel, hz, source?)`: Poisson drive. The rate a neuron sees
is the largest any source asked for, so the fly's senses and the bench's
experiments can drive the same neurons without taking each other's drive
away. `brain:drives()` lists `{ source, count, hz }` per source and
`brain:clearStimulation(source?)` clears one source or all of them.
- `brain:silence(sel, on)`: ablate or restore.
- `brain:watch(name, sel)` / `brain:unwatch(name)` / `brain:rates()`:
read groups out as Hz per neuron. A neuron carries at most three groups at
once and a watch that would need a fourth on any neuron is refused, naming
the neuron and the groups it holds. Watching a name again with the same
selection is a no-op; `unwatch` frees the slots and the group's id.
- `brain:classRates(limit?)`: the cell classes with the most activity right
now, from the 60 ms spike trace. `brain:groupActivity(name)`: that trace
per neuron for a watched group of at most 32, which is what a raster draws a
row from. Both arm a per-neuron readback that is taken at most every twelve
frames and only while something is asking, because it reads the whole
activity lane.
- `brain:activity()`: network mean rate, total spikes, dropped spikes,
simulated time.
- `brain:info(i)`: what one neuron is.
- `brain:buildCloud(opts, onReady?)` / `brain:setCloudDrawn(on)` /
`brain:cloudDrawn()` / `brain:cloudBounds()` / `brain:setCloudOrigin(xyz)` /
`brain:showCloud(opts)` / `brain:hideCloud()` / `brain:worldPosition(i)` /
`brain:neuronAt(screenX, screenY, radiusPx?)`: the instanced neuron cloud.
- `brain:setParams(patch)`, `brain:setRunning(on)`, `brain:setTimeScale(x)`,
`brain:setMaxStepsPerFrame(n)`, `brain:reset()`, `brain:ready()`,
`brain:destroy()`, `brain:describe()`.
## Usage
```luau
local Brain = require("~.brain") -- from inside the package
local brain = Brain.load(asset.resolve("fruitFly.maleCNS", "connectome"), { gain = 0.6 })
brain:watch("MN9", { types = { "MN9" } })
brain:stimulate({ types = { "claw_tpGRN" } }, 100, "bench")
-- every frame:
brain:update(dt)
print(brain:rates().MN9)
```
A selection is `{ types?, typePrefix?, pattern?, classes?, superclasses?,
sides?, excitatory?, indices?, limit? }`; every given key must hold.
## The cloud, and why it is built in two halves
A registered instanced population has its whole transform run recomposed on
the CPU every frame, whether or not the cull draws any of it, so 139,741
neurons cost about 4.7 ms of frame whether or not anyone is looking at the
brain. `buildCloud` fills the buffers and `setCloudDrawn` registers or drops
the draw without touching them, which is what lets `FruitFlyBrain` put the
registration down while the camera is pointed somewhere else and put it back
the frame the cloud enters view. Taking it down leaves `worldPosition`,
`neuronAt` and `cloudBounds` answering as before.
Only neurons that have a soma get a slot, which is 139,741 of 164,587. Slot
and neuron index are therefore different numbers, so the `cloud_gather` kernel
copies each slot's activity out of the neuron it draws once a frame, and
`slotToNeuron` maps back for picking.
The fill is spread over frames from a task, about forty scan slices and
eighteen upload slices, so building the cloud costs about a millisecond a
frame for a second rather than one frame of a third of a second.
## Notes
- The network is silent until something is driven; a spike always traces
back to a stimulus.
- Every GPU buffer is named `brain.<id>.<name>`; `compute.observe()` lists
them. `destroy` releases them all.
- The kernels' `params` are per shader, so two networks on one engine share
the model constants the last `applyParams` wrote.
- Editing this file hot-reloads its methods onto networks that are already
running. A network records the `SHAPE` it was made with and `FruitFlyBrain`
loads a fresh one when the module's shape has moved past it, so raise
`SHAPE` whenever a field is added to or taken off a network or its cloud.
·computeshader · born here
❒asset # cloud_gather
Copies each drawn cloud slot's activity out of the neuron that slot draws.
The `brain` module registers one instance per neuron that has a soma, which is
139,741 of the male CNS's 164,587, so the slots are packed and slot `s` is not
neuron `s`. The integrate kernel writes its activity lane at each neuron's own
index, so without this pass every slot would be lit by the wrong neuron. It
reads `slotmap[s]`, the neuron index the module recorded for that slot when it
built the cloud, and copies the three values of that neuron's lane 0 into
`cloudviz[s]`, which is the buffer the instanced draw hands the neuron
material as `input.shader_data[0]`.
Lane 1 of `cloudviz` is the slot's resting colour, written once from the CPU
at build, and this kernel leaves it alone.
`Brain:step` dispatches it once after the frame's last simulation step, and
only while the cloud's draw is registered.
Bindings: `viz` (read), `slotmap` (read), `cloudviz` (read_write). Param:
`drawn`, the registered instance count. One thread per slot at a workgroup
size of 64.
# connectome
A whole nervous system's wiring diagram as one asset: every neuron with its
soma position, cell type, class and neurotransmitter, and every
neuron-to-neuron connection with its synapse count, packed into the binary
form a GPU spiking simulation binds directly. The `brain` module of the
`fruitFly` package runs any instance of this type; the type itself carries
no animal.
```
maleCNS.connectome/
manifest.json counts, provenance, byte layouts, vocabularies
neurons.bin 32 bytes per neuron
offsets.bin u32[neurons + 1], CSR row starts into edges.bin
edges.bin u32 per connection: target index | synapse count << 18
README.md the animal, the dataset, the release, the license
.metadata description + tags
```
## The manifest
```json
{
"dataset": "MaleCNS v1.0 ...",
"license": "CC-BY 4.0",
"citation": "...",
"neurons": 164587,
"edges": 25563197,
"edge_format": "...",
"neuron_record": "...",
"flags": "...",
"sign_rule": "...",
"soma_centre_um": [x, y, z],
"vocab": {
"types": [...], "superclasses": [...], "classes": [...],
"neurotransmitters": [...], "sides": [...], "dimorphism": [...], "fruDsx": [...]
}
}
```
## The neuron record (32 bytes, little-endian)
| offset | type | field |
|---|---|---|
| 0 | f32 | soma x, micrometres, centred on `soma_centre_um` |
| 4 | f32 | soma y |
| 8 | f32 | soma z |
| 12 | u32 | index into `vocab.types` |
| 16 | u32 | flags (below) |
| 20 | u32 | source body id, low 32 bits |
| 24 | u32 | source body id, high 32 bits |
| 28 | u32 | out-degree (number of edges in this neuron's row) |
Flags: bits 0-5 superclass, 6-13 class, 14-17 neurotransmitter, 18-19 side,
bit 20 has a soma position, 21-23 dimorphism, 24-26 fruDsx, bit 27 set when the
neuron excites its targets (clear when it inhibits). Every bit field indexes
the vocabulary of the same name, 0 being `unknown`.
## Parts
A payload larger than one blob may carry is stored as numbered parts,
`edges.0.bin`, `edges.1.bin`, ..., and the manifest's `parts` says how many
(`"parts": { "edges": 2 }`). `ref:parts("edges")` lists them with the byte
offset each starts at, so a GPU upload writes each part in place;
`ref:bytes("edges")` concatenates them when a caller wants the whole table.
A payload without an entry in `parts` is the single file `<name>.bin`.
## The edge (4 bytes)
Bits 0-17 hold the postsynaptic neuron's index, bits 18-31 the synapse count
clamped at 16383. Row `i` of the table is `edges[offsets[i] .. offsets[i+1])`,
so a neuron's outgoing connections are one contiguous run, which is what an
event-driven spike propagation reads.
## Reading one from Luau
```luau
local ref = asset.resolve("maleCNS", "connectome")
local m = ref:manifest() -- the decoded manifest
local neurons = ref:bytes("neurons") -- a Luau `buffer` of neurons.bin
local edges = ref:bytes("edges") -- the whole edge table, for a GPU upload
ref:counts() -- { neurons = ..., edges = ... }
```
`ref:bytes(name)` reads the whole file into a Luau `buffer`, so a call for
`edges` on a large connectome holds a hundred megabytes until the caller drops
it; hand it to a GPU buffer's `writeBytes` and let it go.
## Making one
A connectome is written by a packer outside the engine (the `fruitFly`
package's README names the script for MaleCNS). It reads the dataset's own
export, assigns dense indices, sorts edges by presynaptic neuron, and writes the
four files above. `asset.create("connectome", "<name>")` scaffolds the folder
with an empty manifest for a packer to fill.
# crackle
The network's own sound: 72 short decaying clicks scattered over two seconds
of mono 24 kHz, synthesised with `audio.encodePcm` and stored as `pcm` so the
loop joins without a click (`loopSeam().ratio` reads 0.0). Every click lands
whole inside the buffer, so the first and last samples are silence.
`FruitFlyBrain` plays it from the Brain entity on a looping spatial `Audio`
component and sets its volume from the whole-network firing rate: silent below
0.05 Hz, rising to a ceiling of 0.35 by 5 Hz. A reset is silence, the taste
experiment's 1.3 Hz is a crackle, the auditory control's 4.6 Hz is a hiss.
# experiments
The bench manual for the `fruitFly` package and the five experiments it was
built for: what a selection is and what each term counts, what a quiet
network means and what recurrence looks like, how a protocol run is made and
how to read its verdict line, and the full control table for keyboard,
gamepad and touch.
Read it with `guides { path = "fruitFly.package/experiments" }`, or open the
experiments panel, which runs four of the five with one press each.
# experiments
The controls of the experiments panel: `expToggle`, `expNext`, `expPrev`,
`expRun` and `expCompare`, each bound on keyboard, gamepad and touch or
saying in its own record which class it deliberately has none of.
| Control | Keyboard | Gamepad | Touch |
|---|---|---|---|
| `expToggle` | P | Select | The "Experiments" button, bottom right |
| `expNext` | N | Right stick press | Tap the protocol |
| `expPrev` | B | None: the list wraps | Tap the protocol |
| `expRun` | R | Right trigger | RUN |
| `expCompare` | C | Left trigger | COMPARE |
`expNext` and `expPrev` are the panel's focus ring: they move the focus
through the five protocols, and the focused protocol is the selected one, so
a pad reaches every protocol without an activate button. The list wraps, which
is why `expNext` alone is enough on a pad and `expPrev` says in its own record
that it has no pad binding: every other button on a pad already carries
something in this world, including the whole D-pad, which is the bench's focus
ring.
The map names no group, so opening the panel leaves the avatar's own controls
live. `FruitFlyExperiments` activates it in `awake` and releases it in
`onDestroy`.
# feed
A wet tick for a mouthful: a 760 Hz blip with a noise transient and a fast
decay, 64 ms long, synthesised in Luau and stored as pcm. `flySound` plays it
once each time the fly's proboscis reaches food, at a ceiling of 0.6.
# flyBody
A fruit fly's body as an entity tree, and a pose for it. `build` stands one
under a root entity; `update` moves the root and poses every part from a motor
command.
The body is a thorax, a head with two compound eyes and a pair of antennae
reaching forward and down, a banded abdomen, a proboscis that extends to feed,
six legs of two segments each walking a tripod gait, and two wings that lie
back over the abdomen at rest and swing out to beat.
```luau
local FlyBody = require("~.flyBody")
local body = FlyBody.build(public.entity, { length = 0.6 })
body.groundY = 0
body.maxRadius = 12 -- the body cannot be moved past this radius
body:update(dt, { speed = 0.3, turn = 0.2, proboscis = 0, flight = 0 })
```
The motor command is `{ speed, turn, proboscis, flight, jump, land }`: speed in
world units per second along the fly's facing, turn in radians per second
(positive turns left), proboscis and flight 0 to 1, `jump` to hop, `land` to
put the body down. Flight lifts the body only while it is already off the
ground, so a beating wing on the floor is a beating wing and the legs keep
walking under it.
Readings: `eye()` returns the head's world position and the facing direction;
`mouthPosition()` the proboscis tip; `rightFrontFoot()`, `leftFrontFoot()` and
`frontFeet()` where the front feet touch; `heading()` the yaw.
`destroy()` takes down only the entities this body built, so anything else
parented to the same root is left standing.
The wing and eye materials are `fruitFly.wing` (a thin, mostly clear
membrane) and `fruitFly.eye` (smooth enough to hold a highlight).
# flySound
The sound a fly makes. `build` hangs three spatial sources under the fly: a
looping wingbeat bed whose loudness follows how hard the wing motor neurons
are driving the wings, a wet tick for a mouthful, and a dry thump for a hop.
```luau
local FlySound = require("~.flySound")
local sound = FlySound.build(public.entity)
sound:setFlight(dt, motor.flight)
sound:feed()
sound:jump()
```
Every clip is pcm and was synthesised from arithmetic in
`fruitFly.wingbeat`, `fruitFly.feed` and `fruitFly.jump`, so the rig needs no
network and no generation service. The wingbeat is a 220 Hz fundamental with
three harmonics over exactly 44 periods, which is what makes it loop without a
click; `clipRef:loopSeam().seamless` reads true on it.
Mix ceilings: wingbeat 0.5, tick 0.6, thump 0.6. The fly is heard out to 14
world units. `volumes()` reports what each source is set to right now, which
is the reading to check the bed against `motor.flight`.
A source with no clip is left out rather than played silent, so a world that
has not got the clips yet runs with the rest of the fly intact.
# jump
A dry thump for a hop: a 92 Hz body with a click on the front, 120 ms long,
synthesised in Luau and stored as pcm. `flySound` plays it once each time a
giant-fibre burst makes the fly jump, at a ceiling of 0.6.
# lab.inputMap
The Fruit Fly Lab bench's control scheme. It carries seven controls, each
bound on keyboard, gamepad and touch (or deliberately suppressed on a class
where a finger already does the thing directly):
| Control | Keyboard | Gamepad | Touch |
|---|---|---|---|
| `labToggle` | L | Start | "Lab" button |
| `labFocusNext` | Down arrow, Tab | D-pad down | tap the control |
| `labFocusPrev` | Up arrow | D-pad up | tap the control |
| `labActivate` | Enter | North | tap the control |
| `labAdjustLeft` | Left arrow | D-pad left | drag the slider |
| `labAdjustRight` | Right arrow | D-pad right | drag the slider |
| `labIdentify` | Mouse left | West | "ID" button |
The map declares no `group` and no `suppresses`, so it composes with the
player's walking scheme instead of standing it down: you can walk at the fly
with the bench open.
`FruitFlyLab.component` activates it in `awake` and releases the claim in
`onDestroy`.
·computeshader · born here
❒asset # lif_clear
Closes one step of the `brain` module's network: empties the spike list the
integrate kernel filled and the propagate kernel consumed, and advances the GPU
step clock by one. The clock living on the GPU is what lets the module queue
several steps in one frame, each reading its own ring slot, with no CPU write
between dispatches.
·computeshader · born here
❒asset # lif_integrate
The membrane step of the `brain` module's leaky integrate-and-fire network,
one thread per neuron. It drains this step's slot of the fixed-point input ring
into the neuron's synaptic conductance, advances the membrane by exponential
Euler on `dv/dt = (vRest - v + g) / tauM` and `dg/dt = -g / tauSyn`, holds a
refractory neuron at reset, draws the neuron's external Poisson event from its
`stim` rate, and on a spike resets it, appends its index to this step's spike
list, and bumps the total and its readout group's counter. Every neuron writes
its activity trace, normalised membrane and spike flag into lane 0 of the
visualisation buffer the neuron cloud draws from. `decayMem` and `decaySyn`
are `exp(-dt / tau)` computed once by the module; the step index comes from the
GPU clock so several steps can be queued in one frame with no CPU write between
them.
·computeshader · born here
❒asset # lif_propagate
The synaptic step of the `brain` module's network. Each workgroup takes one
spike from this step's list, and its 64 threads stride that neuron's outgoing
edge row, adding every connection's synapse count times `unitsPerSynapse`
(signed by the presynaptic neuron's transmitter) into the fixed-point input
ring at the slot `delaySteps` after the current step. A connection carrying fewer than
`minSynapses` synapses is skipped, which is how the module applies the
connection threshold whole-brain models are fitted with (five, in Shiu et al.
2024) without repacking the table. The module dispatches it with `maxSpikes`
workgroups; a workgroup past this step's spike count returns at once. The sum is integer atomics, so the result is the same whatever order
the threads land in.
·computeshader · born here
❒asset # lif_reset
Puts the `brain` module's network at rest in one dispatch: every membrane at
`vRest` with no synaptic conductance, refractory time or activity trace, every
slot of the input ring empty, the visualisation lane dark, the spike list
empty, every spike counter zero and the step clock at zero. The module runs it
once after allocating a network and again on `brain:reset()`.
# maleCNS
The complete central nervous system of an adult male *Drosophila melanogaster*:
brain, both optic lobes and the ventral nerve cord, 164,587 traced neurons and
25,563,197 neuron-to-neuron connections carrying 125 million synapses.
Packed from the MaleCNS v1.0 flat-connectome export (HHMI Janelia FlyEM,
Google Research and the Cambridge connectomics group; released 8 June 2026,
published as Berg et al. 2026, *Sexual dimorphism in the complete Drosophila
male central nervous system connectome*, Cell, doi:10.1016/j.cell.2026.08.015).
The export used is the `traced-only`, minimum-confidence 0.5 connectivity table,
with the curated body annotations (cell type, class, superclass, side, sexual
dimorphism, fru/dsx expression) and the per-body consensus neurotransmitter
prediction. Soma positions come from the annotation table's soma location in
8 nm voxels, converted to micrometres and centred; 139,741 neurons carry one,
the rest sit at the origin with the has-soma flag clear.
Data license: CC-BY 4.0, https://male-cns.janelia.org/. The manifest carries
the SHA-256 of each source file.
Sign convention: a neuron whose consensus transmitter is GABA, glutamate or
histamine inhibits every target; every other transmitter excites (Shiu et al.
2024, Nature). Connection strengths are synapse counts; the wiring says which
neurons reach which and how many synapses they make, and nothing about the
efficacy of each synapse in a living fly.
# neuron
The surface of one neuron in the brain's instanced neuron cloud. It lights
itself from the instance data lane 0 that the `cloud_gather` kernel fills each
frame with the activity of the neuron this slot draws (activity trace,
normalised membrane, spike flag) and the superclass colour the module writes
once (lane 1): dim at rest, brighter as the membrane depolarises, flaring well
above 1 on a spike and fading with the trace, which is where the scene's bloom
picks it up.
`rest_brightness`, `membrane_brightness`, `spike_brightness` and
`rim_brightness` set those four levels on the material.
# protocol
The experiments from `experiments.guide` as records anything can run, and the
verdict a run computes from its own numbers.
`Protocol.all()` is the five. `Protocol.variants(proto)` turns one into its
runnable forms, one per variant. `Protocol.run(brainComp, proto, onDone)`
starts a run and `Protocol.tick(run)` advances it, once a frame, until it
reports itself done. Nothing in here spawns a task, so a run cannot outlive
whoever is ticking it.
```luau
local Protocol = require("~.protocol")
local comp = entity.find("Brain").component.get("FruitFlyBrain")
local form = Protocol.variants(Protocol.byId("taste"))[1]
local run = Protocol.run(comp, form)
-- from update(dt), every frame:
if Protocol.tick(run) then
print(run.verdict)
-- MN9 peaked 130 Hz at 2497 ms, held 112 Hz; Kenyon_Cell 0.023 Hz;
-- network mean 1.30 Hz, 1.30 over baseline (confined)
end
```
A run starts from rest: it clears every drive and resets every membrane, so
the state the network was left in cannot reach its numbers. It then records
`Protocol.BASELINE_MS` of the network before its first drive: the fly is alive
while a protocol runs and what it drives through its own sensory neurons is
part of the network's rate, and the readouts need about a second of the
network's own clock before they report the reset network rather than the one
before it. The verdict's spread word is computed
on the rate the drive ADDED to that baseline, so an experiment reads the same
whether the fly is standing still or seeing the arena wall loom. A protocol's
`durationMs` is the length of its drive phase; the lead-in is on top of it,
and a peak's time is reported from the moment the drive started. It applies its own
gain and minimum synapse count for the length of the run and puts the previous
values back. It silences what it ablates and restores it. It stops its own
drives by driving them at 0 Hz, so no other drive is touched.
Readouts are watched under the names the guide uses (MN9, Kenyon_Cell, DNp01,
LPLC2, LC4), so a bench watching the same cell types under the same name
shares the group rather than spending a second readout slot on each neuron.
`FruitFlyExperiments` is the panel that runs these, and `FirstRun` is the card
that walks a newcomer to the first result. Both require this module, and the
`Protocol.ui` table is how they talk to each other.
# selection.module
The grammar the lab bench, the experiments panel and the guide all read when
a person names a set of neurons. A selection is a line of space separated
`key:value` terms, and it turns into the table `brain:select` takes.
| Term | Selection key | Shape |
|---|---|---|
| `type:MN9,DNa01` | `types` | comma list, merged across repeated terms |
| `prefix:JO-` | `typePrefix` | one value, last term wins |
| `pattern:^LB%d` | `pattern` | one value, a Luau pattern over the type name |
| `class:Kenyon_Cell,MBON` | `classes` | comma list, merged |
| `super:descending_neuron` | `superclasses` | comma list, merged |
| `side:L` | `sides` | comma list, merged |
| `excitatory:true` | `excitatory` | true or false |
| `indices:0,1,2` | `indices` | comma list of neuron indices |
| `limit:100` | `limit` | a number |
`class:`, `classes:`, `super:`, `superclass:`, `superclasses:`, `side:`,
`sides:`, `type:`, `types:`, `index:` and `indices:` are all accepted
spellings of the same keys.
Every key given must hold, so terms of different keys intersect and terms of
the same key union: `type:MN9 type:DNa01` names four neurons,
`class:Kenyon_Cell side:L` names the left Kenyon cells.
## Reading it
```lua
local Selection = require("~.selection")
local sel, err = Selection.parse("type:MN9,DNa01 side:L")
if sel == nil then
print(err) -- names the term that failed
else
print(Selection.describe(sel)) -- "types MN9, DNa01; sides L"
print(#brain:select(sel))
end
local brainComp = Selection.resolve(public.brain) -- the FruitFlyBrain, or nil
```
`M.PLACEHOLDER` is the field placeholder that names every term, and `M.KEYS`
is the same list as one line for an error message, so the field, the error
and this table cannot drift apart.
# wingbeat
The fly's wingbeat bed. A 220 Hz fundamental with three harmonics and a slow
amplitude wobble, 0.2 s long, which is exactly 44 periods of the fundamental
and one of the wobble, so the clip's last frame runs into its first without a
click (`loopSeam().seamless` reads true). Synthesised in Luau and stored as
pcm, so nothing about it depends on a network or a generation service.
`flySound` plays it looping and spatial on the fly, with its volume following
how hard the wing power-muscle motor neurons are driving the wings, up to a
ceiling of 0.5.
# expCompare
Overlays the run before the last one, dimmed, on the panel's trace.
# expNext
Moves the panel's focus to the next protocol, wrapping at the end. The focused
protocol is the selected one, so this is both the focus ring and the choice.
# expPrev
Moves the panel's focus to the previous protocol, wrapping at the start. It
has no pad binding: the list wraps, so `expNext` alone reaches every protocol
on a pad.
# expRun
Runs the selected protocol, or stops the one running.
# expToggle
Shows or hides the experiments panel. Bound on every device class: the touch binding is the panel's own "Experiments" button in the bottom-right zone.
# labActivate
Presses the bench control the focus is on: a button fires, a text field takes
the keyboard. Enter on a keyboard, the north face button on a pad. Touch is
suppressed: a tap on the control is the activation.
# labAdjustLeft
Moves the focused slider down by 5 percent of its range: gain, time scale,
minimum synapses, or the stimulation rate. Touch is suppressed because a
finger drags the track itself.
# labAdjustRight
Moves the focused slider up by 5 percent of its range. The mirror of
`labAdjustLeft`.
# labFocusNext
Moves the bench's focus one control down its order, so the whole bench is
reachable from a pad or a keyboard with no pointer. The focused control
draws a 2 px accent outline. Touch is suppressed on purpose: a finger taps
the control directly.
# labFocusPrev
Moves the bench's focus one control up its order. The mirror of
`labFocusNext`; touch is suppressed for the same reason.
# labIdentify
Names the neuron in the cloud under the pointer, and fills the bench's
Neuron block with its cell type, class, superclass, side, transmitter and
out degree. With no pointer on the surface (a pad or the touch button) it
names the neuron under the screen centre instead. A press the bench's own
window consumed is ignored, so clicking a button is never also a pick.
# labToggle
Shows or hides the Fruit Fly Lab bench window. L on a keyboard, Start on a
pad, the "Lab" button on a touch screen. The bench is the only way to drive
the network by hand, so this control is on the pad's menu button rather than
a face button a player uses for something else.