# fruitFly
A whole fruit fly nervous system running as a spiking network on the GPU, a
fly body it drives, and a bench for experiments on it. Install this package
and a world holds the complete adult male *Drosophila* central nervous system
(MaleCNS v1.0: 164,587 neurons, 25.6 million connections, CC-BY 4.0) as a
leaky integrate-and-fire network you can stimulate, silence, read out and
watch, plus a fly whose taste, looming, walking, turning, feeding, flight and
escape run through that wiring.
## Contents
The network:
- `maleCNS.connectome/` is MaleCNS v1.0 packed for the GPU (109 MB, the edge
table in two parts). It is an instance of the `connectome` asset type, which
lives at the world root (`/zero/source/connectome.assetType/`) and travels
with the package as a dependency: any nervous system's wiring as one asset
(`manifest.json`, `neurons.bin`, `offsets.bin`, `edges.bin` or its parts).
`guides { path = "types/connectome" }` is the reference.
- `lif_integrate`, `lif_propagate`, `lif_clear`, `lif_reset` (`.computeShader`)
are the four kernels of the network: membrane step, event-driven spike
propagation over the CSR edge table, step close, reset.
- `brain.module/` is `Brain.load(connectomeRef, params)`, which binds a
connectome to the kernels and returns the network: `step`, `update`,
`select`, `stimulate`, `silence`, `watch`, `rates`, `activity`, `classRates`,
`groupActivity`, `drives` and the cloud.
- `FruitFlyBrain.component/` puts a network on an entity: it loads the
connectome the `connectome` field names, steps it every frame, draws the
cloud, and exposes `stimulate` / `silence` / `watch` / `rates` / `reset`
(the first three are synced, so every peer's copy receives the same inputs).
- `neuron.shader/`, `neuron.material/` and `cloud_gather.computeShader/` are
the instanced neuron cloud, each neuron lit by its own activity.
- `crackle.soundClip/` is the network's own sound: a sparse crackle that is
silent when the network is silent and a hiss when it runs away.
The fly:
- `flyBody.module/` is the fly's body as an entity tree with a tripod gait,
wings, proboscis and a hop.
- `FruitFly.component/` is the embodied fly: it reads the world into the
brain's sensory neurons and reads motor commands back out of named cell
types.
- `FoodPatch.component/` is a disc of food on the ground the fly can taste.
- `wing.material/`, `eye.material/` and the `wingbeat`, `feed` and `jump`
sound clips are what the fly looks and sounds like.
The instrument:
- `FruitFlyLab.component/` is the bench window: network state and knobs, live
readouts with traces and rasters, hot classes, live drives, a neuron panel,
a legend, and a selection line to stimulate, silence, restore, count or
watch any set of neurons.
- `selection.module/` is the selection grammar the bench and everything else
parse with.
- `protocol.module/` is the guide's experiments as records anything can run,
and the verdict a run computes from its own numbers.
- `FruitFlyExperiments.component/` is the experiments panel: those protocols
with one press each, a run list, a compare overlay and a verdict line.
- `FirstRun.component/` is the four-step card that walks a newcomer from a
reset network to a proboscis coming out.
- `lab.inputMap/` and `experiments.inputMap/` are the bench's and the panel's
controls, each binding keyboard, gamepad and touch.
- `experiments.guide/` is the bench manual and the five experiments.
The lab around it:
- `labFloor.material/` and `FruitFlyLook.component/` are the floor, the fog,
the vignette and the lighting intent, held as data rather than as code.
- `arenaRing.mesh/` is the ring on the floor the fly stays inside.
## Usage
```luau
-- The brain: one entity, one network. The default connectome is maleCNS.
local brainEnt = entity.spawn("Brain")
brainEnt.position = { 0, 6, -8 }
brainEnt.component.add("FruitFlyBrain", { gain = 0.6, timeScale = 1 })
-- An embodied fly driven by it, and something for it to taste.
local fly = entity.spawn("Fly")
fly.component.add("FruitFly", { brain = brainEnt, bodyLength = 0.6 })
local food = entity.spawn("Food")
food.position = { 3, 0, 0 }
food.component.add("FoodPatch", { radius = 0.6, sugar = 1 })
-- The bench, the experiments panel and the card.
entity.spawn("Lab").component.add("FruitFlyLab", { brain = brainEnt, fly = fly })
entity.spawn("Experiments").component.add("FruitFlyExperiments", { brain = brainEnt, fly = fly })
entity.spawn("FirstRun").component.add("FirstRun", { brain = brainEnt, fly = fly })
-- Or from code:
local comp = brainEnt.component.get("FruitFlyBrain")
comp:watch("MN9", { types = { "MN9" } }) -- proboscis motor neurons
comp:stimulate({ types = { "claw_tpGRN" } }, 100) -- tarsal taste pegs at 100 Hz
print(comp:rates().MN9) -- Hz per neuron, updated every frame
```
A selection names neurons by the dataset's own vocabulary: `types` (exact
cell-type names), `typePrefix`, `pattern` (a Lua pattern over the type name),
`classes`, `superclasses`, `sides` (`L`/`R`/`M`), `excitatory`, `indices`,
`limit`. `brain:info(i)` answers what any neuron is.
## The model
Every neuron is a leaky integrate-and-fire unit with the constants Shiu et
al. 2024 (Nature) fitted to the FlyWire brain: rest and reset at -52 mV,
threshold -45 mV, membrane time constant 20 ms, synaptic time constant 5 ms,
refractory period 2.2 ms, transmission delay 1.8 ms, 0.275 mV per synapse. A
presynaptic neuron whose consensus transmitter is GABA, glutamate or histamine
inhibits; every other transmitter excites. A connection carries its synapse
count; connections under `minSynapses` (default 5) carry nothing.
`gain` scales every synapse. MaleCNS traces about 2.5 times the synapses per
connection FlyWire does, so at gain 1 the network runs away under any drive;
at 0.6 (the default) driving the 50 tarsal taste-peg neurons at 100 Hz holds
the proboscis motor neurons MN9 at about 115 Hz while the Kenyon cells stay
under 0.1 Hz, and driving the 595 Johnston's organ neurons instead leaves MN9
at zero. At 0.5 the taste drive reaches nothing. The bench's gain slider is
the experiment, and the experiments panel's "Gain recovery" runs all three in
one press.
Everything runs on the GPU: the state, the input ring, the spike list and the
edge table. The CPU sees the per-group spike counters each frame and, on
request, the per-neuron activity lane. Integer atomics make a step
order-independent, so two peers stepping the same inputs read the same
network.
## The fly
`FruitFly` reads named cell types out of the brain and writes the world into
others; every name is in `FruitFly.component/init.luau`:
| command | cell types |
|---|---|
| walk | the leg motor neurons (`vnc_motor` types ending in ` MN`), per side |
| turn | `DNa01`, `DNa02` left minus right |
| proboscis | `MN9` |
| flight | the wing power-muscle motor neurons (`DLMn`, `DVMn`) |
| jump | a `DNp01` (giant fibre) burst |
| sense | cell types | driven by |
|---|---|---|
| tarsal taste | `claw_tpGRN`, `dorsal_tpGRN`, per side | that side's front foot on a `FoodPatch` |
| labellar taste | `LB*` labellar bristle GRNs | the extended proboscis on a `FoodPatch` |
| looming, LPLC2 path | `LPLC2` | a player's avatar growing fast in front of the fly |
| looming, LC4 path | `LC4` | the same, driven as a separate path so one can be silenced |
The two looming paths are driven separately and weighted by how head-on the
approach is, so an avatar walking up behind the fly does not loom and
silencing one detector class is an experiment with an answer.
The body moves kinematically at `walkSpeed` body lengths per second; the fly
faces its entity's -Z. On a shared world the entity's owner writes the senses
into the brain (a synced call) and every peer poses its own copy from its own
network.
## Controls
| Action | Keyboard | Gamepad | Touch |
|---|---|---|---|
| Bench: show or hide | L | Start | The "Lab" button |
| Bench: move focus | Down and Up, Tab | D-pad down and up | Tap the control |
| Bench: activate the focused control | Enter | North (Y) | Tap |
| Bench: nudge the focused slider | Left and Right | D-pad left and right | Drag it |
| Bench: identify a neuron | Click it in the cloud | West (X) | Tap it |
| Experiments: show or hide | P | Select | The "Experiments" button |
| Experiments: next protocol | N | Right stick press | Tap the protocol |
| Experiments: previous protocol | B | None: the list wraps | Tap the protocol |
| Experiments: run | R | Right trigger | RUN |
| Experiments: compare | C | Left trigger | COMPARE |
Both maps name their controls apart (`lab*` and `exp*`) and take disjoint
keys and buttons, so the bench and the panel can be open together and neither
answers for the other.
## Data and license
MaleCNS v1.0: HHMI Janelia FlyEM, Google Research and the Cambridge
connectomics group, released 8 June 2026 (Berg et al. 2026, Cell,
doi:10.1016/j.cell.2026.08.015), CC-BY 4.0. The pack was written by a script
that reads the public flat-connectome export (`connectome-weights ... traced-only`,
body annotations, consensus neurotransmitters), assigns dense indices, sorts
edges by presynaptic neuron and writes the four files `types/connectome`
describes; the manifest carries each source file's SHA-256.
## Notes
- Loading the network reads the 102 MB edge table into a GPU buffer once per
`FruitFlyBrain`; the VFS copy is released afterwards.
- The neuron cloud is one instanced draw of every neuron with a soma
(139,741). Its transforms are static; the activity lanes are written by the
integrate kernel.
- `maxStepsPerFrame` bounds GPU time per frame; at 32 steps of 1 ms a 60 fps
engine keeps up with real time.
- A neuron reads out under at most three groups at once. The fly holds seven
of its own and the protocols watch five more under the guide's names, so a
bench watch of the same cell types under the same name shares the group
rather than spending another slot.
# flyBody
A fruit fly's body as an entity tree, and a pose for it. `build` stands one
under a root entity; `update` moves the root and poses every part from a motor
command.
The body is a thorax, a head with two compound eyes and a pair of antennae
reaching forward and down, a banded abdomen, a proboscis that extends to feed,
six legs of two segments each walking a tripod gait, and two wings that lie
back over the abdomen at rest and swing out to beat.
```luau
local FlyBody = require("~.flyBody")
local body = FlyBody.build(public.entity, { length = 0.6 })
body.groundY = 0
body.maxRadius = 12 -- the body cannot be moved past this radius
body:update(dt, { speed = 0.3, turn = 0.2, proboscis = 0, flight = 0 })
```
The motor command is `{ speed, turn, proboscis, flight, jump, land }`: speed in
world units per second along the fly's facing, turn in radians per second
(positive turns left), proboscis and flight 0 to 1, `jump` to hop, `land` to
put the body down. Flight lifts the body only while it is already off the
ground, so a beating wing on the floor is a beating wing and the legs keep
walking under it.
Readings: `eye()` returns the head's world position and the facing direction;
`mouthPosition()` the proboscis tip; `rightFrontFoot()`, `leftFrontFoot()` and
`frontFeet()` where the front feet touch; `heading()` the yaw.
`destroy()` takes down only the entities this body built, so anything else
parented to the same root is left standing.
The wing and eye materials are `fruitFly.wing` (a thin, mostly clear
membrane) and `fruitFly.eye` (smooth enough to hold a highlight).
# FruitFlyBrain
A whole connectome running as a spiking network on this entity. It loads the
`connectome` asset the field names (the package's `maleCNS` by default),
steps it every frame the gameplay clock runs and, with `runInEdit`, in edit
mode too; draws every neuron with a soma as a lit point around the entity;
plays the network's own crackle from it; and hands the live network to
anything that asks.
Fields: `connectome`, `running`, `runInEdit`, `timeScale`,
`maxStepsPerFrame`, `gain`, `minSynapses`, `showCloud`, `cloudGate`,
`cloudScale`, `neuronRadius`, `bloom`, `bloomIntensity`, `crackle`. The
defaults are the calibrated regime: gain 0.6, minSynapses 5, timeScale 1,
maxStepsPerFrame 32.
Methods on the ref: `getBrain()`, `stimulate(sel, hz, source?)`,
`silence(sel, on)`, `clearStimulation(source?)`, `drives()`, `reset()`,
`watch(name, sel)`, `unwatch(name)`, `rates()`, `classRates(limit?)`,
`groupActivity(name)`, `neuronAt(screenX, screenY, radiusPx?)`,
`keepCloudDrawn(seconds?)`, `describe()`. `stimulate`, `silence`,
`clearStimulation` and `reset` are synced calls, so every peer's copy of the
network receives the same inputs. `onLoaded` fires with the neuron and edge
counts once the network is on the GPU.
```luau
local e = entity.spawn("Brain")
e.position = { 0, 5, -8 }
e.component.add("FruitFlyBrain", { gain = 0.6 })
local c = e.component.get("FruitFlyBrain")
c:watch("MN9", { types = { "MN9" } })
c:stimulate({ types = { "claw_tpGRN" } }, 100, "bench")
```
## The visibility gate, and what it means for a capture
The cloud is 139,741 instances, and a registered instanced population has its
whole transform run recomposed on the CPU every frame whether or not the cull
draws any of it. Measured on this world: `render.mesh_transform_compose` is
4.74 ms a frame with the cloud registered and 0.67 ms with it down. So the
component tests the cloud's bounding sphere against the on-screen camera's
frustum every three frames, registers the draw the moment the sphere enters,
and drops it after thirty consecutive frames outside. `showCloud = false`
still takes everything down.
The gate only knows about the camera on screen. **An offscreen capture aimed
at the brain from a station the player is not standing at can therefore come
back with no cloud in it.** Call `keepCloudDrawn(seconds)` first, or set
`cloudGate` false.
Moving the entity moves the cloud with it: the transforms are rewritten from
the neuron records a slice a frame, with no rebuild.
## Drives are owned
`stimulate` takes the name of whoever is driving. The rate a neuron sees is
the largest any source asked for, so the fly stepping onto a food patch
(`"fly"`) cannot take away what the bench (`"bench"`) or a protocol
(`"protocol"`) is driving, and `clearStimulation("fly")` clears only the
fly's. A call that names no source is recorded as `"code"`. `drives()` is what
the bench lists.
## Sound and light
While the cloud is drawn the component registers the builtin bloom on the
`scene` layer, which is what turns a spiking neuron from a bright dot into a
point of light, and takes it off again when the cloud goes down. The Audio
component it puts on the entity plays the package's `crackle` clip on a loop
with its volume following the whole-network rate: silent below 0.05 Hz,
rising to 0.35 by 5 Hz.