audio Module

Public Luau surface over the __audio Internal FFI namespace — the engine-native CPU audio codec for the canonical ZAUD payload (backed by zero_audio::blob), and the engine's audio observation surface. Auto-injected as _G.audio via the prelude.

Observing what is audible

audio.observe() answers "what is this engine sounding right now, and why is a source not" in one read. It reports the MIXER's own bookkeeping, which is what makes it different from reading the Audio component's fields back: those carry what content asked for, and AudioSource.state carries the value :play() wrote there. Every voice report carries both — mixerState beside componentState — so the two are compared rather than confused.

• audio.observe() -> AudioObservation — the whole surface in one call: { available, sampleRate, channels, sampledAtMs, voices, audibleCount, listener, voiceAccounting, levels, profile }.
• audio.voices() -> { AudioVoice } — every live source with the mixer's opinion of it: the clip, the channel, whether its bytes are resident, the mixer's playback state, the position in the clip and the clip's duration, the gain it reaches the mix at stage by stage, the rate it plays at (pitch, the Doppler ratio doppler and their product rate), and for a spatial source its emitter, the listener distance, maxDistance, rolloff and the radialSpeed it and the listener move apart at.
• audio.voice(entityId) -> AudioVoice? — the voice on one entity.
• audio.whySilent(entityId) -> reason?, detail? — why a source makes no sound, from a closed set: noBackend, noDevice, notResident, neverStarted, refused, paused, ended, gainZero, channelSilent, masterSilent, outOfRange. nil while it IS sounding; the second return carries the mixer's own words when it refused the source.
• audio.device() -> AudioDeviceStatus — the output device the mix reaches: state ("open" / "silent"), the device an open stream runs on, the rate and channel count it runs at, and the counters for what the engine has been through keeping a stream open — faults, changes, reopens, failedOpens, glitches and lastError. An output device that goes away leaves the mixer running and every voice reporting noDevice until the engine opens a stream again. parked is true while nothing plays and the output rests (parks counts the times, resumeLatencyMs is how long the last resume took to play).
• audio.levels() -> AudioLevels — the master mix's peak and RMS over the meter's most recent closed window, with no recording involved. windows counts the windows the mixer has closed since it started, so a silent mix reads differently from a mixer that is not running.
• audio.peakSince(window) -> number? — the loudest peak the master mix reached across the windows that closed after audio.levels().windows stood at window. levels() carries the window that closed last, so a reader sees the windows its own frames land on; this spans all of them, which is what measuring a sound shorter than the gap between two reads takes. nil when the meter holds no peak for the span — nothing has closed since window, or it reaches further back than the meter's history of recent windows.
• audio.listener() -> AudioListenerState — where the scene is heard from, and how many active listeners exist.
• audio.voiceAccounting() -> AudioVoiceAccounting — voice capacity, in use and free read off the mixer's own tracks, so free is what the next play call is granted or refused against. The mixer keeps two pools and each is reported apart: capacity / inUse / free are the main track, carrying the NON-spatial voices, while a spatial voice plays through its own sub-track and is counted by spatialCapacity / spatialInUse. sourcesHolding counts both pools from the sources that own them, so it equals inUse + spatialInUse while every voice the mixer carries answers to a source — and a mixer carrying voices no source reaches is the difference between those numbers.
• audio.profile() -> AudioProfile / audio.resetProfile() — what the audio subsystem costs: the streaming pump, clip decode and encode, voice starts, and building the observation. Every total is a sum across a window rather than a per-frame figure: the window runs from the last resetProfile — or from engine start when nothing has reset it — and its length comes back as windowMs and frames. Reset, let frames pass, then divide a total by frames for what a frame costs.

The same document is served at /runtime/active_audio, with one file per source under /runtime/active_audio/voices/<entity>, as the audio systems published it after the last frame; audio.observe() leaves out the voices of entities a script has despawned since. It answers in edit mode as well as play mode.

Setting what the mix is heard at

Every voice names a channel — the Audio component's channel field, "sfx" unless the source says otherwise — and is heard at its own volume scaled by the level of that channel and by the master level of the mix. Both levels reach voices that are already playing, so a settings-screen slider is heard as it moves, and both come back per voice as gain.channel and gain.master.

• audio.mixer() -> AudioMixerLevels — { master, muted, channels }. A channel absent from channels has had no level set and plays at unity, so a scene that invented its own channel names is heard exactly as authored until something turns one of them down.
• audio.setChannelVolume(channel, volume) — set one channel's level. Scales every voice on that channel and nothing else.
• audio.setMasterVolume(volume) — set the level of the whole mix.
• audio.setMuted(muted) — silence the mix whatever the levels read; unmuting hands them back untouched.

audio.setChannelVolume("music", 0.3)   -- music at 30%, sfx untouched
audio.setMasterVolume(0.8)
audio.setMuted(true)                   -- every voice reports masterSilent

Playing sound a script synthesizes

audio.stream(entity, options?) -> AudioStream?, err? opens a PCM stream and plays it through the entity's voice: an ecs.AudioSource naming the stream in its stream field, so placement, distance attenuation, Doppler, pitch, value bindings and the mixer's levels apply to it as to any source. options shapes the PCM (sampleRate 4000 to 192000 Hz, default the mixer's rate; channels 1 or 2; format "f32" or "i16"; latencyMs, default 100; bufferMs) and the voice (volume, pitch, spatial, maxDistance, rolloff, doppler, channel, sourceType, pitchValue, volumeValue, autoplay).

• stream:push(pcm, frames?) -> number?, err? — queue interleaved samples (a buffer or string in the stream's format, or an array of numbers); returns the whole frames taken.
• stream:wanted() -> number — the frames that bring the queue up to its target: latencyMs, or one and a half times the longest recent gap between pushes when that is longer.
• stream:status() -> AudioStreamStatus? — queuedFrames, queuedMs, capacityFrames, targetFrames, producerGapMs, wantedFrames, framesPushed, framesRefused, framesPlayed, underruns, starvedFrames, voice, rate; nil once closed.
• stream:play(), stream:pause(), stream:set(settings), stream:flush(), stream:close().

The mixer reads the stream at its own rate, whatever the mixer's, and counts each time it finds the queue empty as an underrun. The stream lives while the entity's audio source names it: removing the source, despawning the entity, putting another source in its place or rewriting the source's stream field closes it.

Codec functions

• audio.encode(sourceBytes, opts?) -> bytes?, err? — decode container audio (ogg / mp3 / wav / flac) and re-encode it into a ZAUD payload.
• audio.encodePcm(pcm, sampleRate, channels, opts?) -> bytes?, err? — encode raw interleaved f32 PCM (a buffer or a binary string of little-endian f32, OR a flat number array) into a ZAUD payload.
• audio.decode(zaud) -> pcm?, sampleRate?, channels? — decode a ZAUD payload to interleaved f32 PCM (pcm is a binary little-endian-f32 string).
• audio.info(zaud) -> table?, err? — read a ZAUD header: { sampleRate, channels, frames, durationMs, codec, loopStart, loopEnd, packets }.
• audio.loopSeam(zaud) -> table?, err? — measure what a clip's samples do where a whole-clip loop wraps: { ratio, step, meanStep, maxStep, channel, seamless, threshold, channels }.
• audio.peaks(zaud, buckets) -> { number }?, err? — the loudest sample in each of buckets (an integer from 1 to 4096) equal spans of a clip, across its channels, held to 1: the shape a waveform is drawn from. A span is frames // buckets frames; the payload is decoded a block at a time, so the work never holds the clip's whole PCM. A payload that is not a clip, or stops short, answers (nil, err).

Every sample an encode is handed must be a finite number. A NaN or an infinity carries no signal, and both codecs turn one into a payload whose header reads as a healthy clip and whose samples sound as nothing — so audio.encode and audio.encodePcm return (nil, err) naming how many samples fail and where the first one sits, and audio.info reads a PCM payload's samples the same way and answers for what the payload carries rather than for what its header claims. That makes a divergent filter, a 0/0, or a math.log(0) in a bake visible at the call that would otherwise have written it.

local zaud, err = audio.encodePcm(samples, 22050, 1, { codec = "pcm" })
if not zaud then
    print(err) -- "…: 3 of 176400 samples are not finite, first at index 88214"
end

Whole-clip repeat is the loop the runtime plays, so the join between a clip's last frame and its first is heard once a lap, and a bed whose head does not meet its tail ticks for as long as it plays. audio.loopSeam is the engine's reading of that join: the step the wrap makes (|x[1] - x[frames]|) against the step the signal ordinarily makes between neighbouring samples, as ratio = step / meanStep. Reading it as a ratio puts the figure in the units the signal itself moves in, so a generated ambience travelling a thousandth of a unit per sample and a synth drone travelling a hundredth are judged the same way. A bed whose partials complete a whole number of cycles across the buffer reads near 1; one carrying a percussive strike at its head and silence at its tail reads in the tens. seamless is ratio <= threshold, and threshold is the ratio asset.create("soundClip", ...) warns past.

local seam = audio.loopSeam(clipRef:getBytes())
print(seam.ratio, seam.seamless, seam.threshold)  -- 41.0  false  8
-- `ratio` is the worst channel's; `channel` names it and `channels` carries
-- each one's own { ratio, step, meanStep, maxStep }.
print(seam.channel, seam.channels[seam.channel].step)

The reading is taken on the DECODED samples, so it answers for what the codec left behind rather than for the buffer that was handed to the encoder, and for a clip that arrived already encoded and whose source buffer nobody holds. That costs a decode of the whole payload — audio.info reads a header without one.

A clip is counted in frames — channels samples that sound together — and a header's frames * channels is the sample count audio.decode hands back. So the buffer audio.encodePcm takes is a whole number of frames, and one whose length leaves a tail over comes back as (nil, err) naming the whole frames it holds and the samples past them.

local zaud, err = audio.encodePcm(fiveSamples, 22050, 2, { codec = "pcm" })
if not zaud then
    print(err) -- "…: 5 samples is not a whole number of 2-channel frames — 2 whole frames and 1 left over"
end

A byte payload carries a sample per 4-byte lane, so audio.encodePcm takes the buffer microphone.samples() hands back and the string audio.decode returns as they are, and recording a capture into a clip is those two calls:

local status = microphone.status()
local zaud = audio.encodePcm(microphone.samples(), status.sampleRate, 1)

A byte length that stops partway through a lane describes a sample the caller holds only part of, and comes back as (nil, err) naming the whole samples it holds and the bytes past them:

local zaud, err = audio.encodePcm(twentySevenBytes, 22050, 1, { codec = "pcm" })
if not zaud then
    print(err) -- "…: 27 bytes is not a whole number of f32 samples — 6 whole samples and 3 bytes left over"
end

opts (all optional): codec ("opus" default / "pcm"), bitrateKbps (default 96), vbr (default true), sampleRate (resample target, default 0 = keep source), forceMono (default false), loopStart / loopEnd (frame indices, default 0). Pure CPU — runs on native, WASM, and headless.