frameStream Module

Public Luau surface over the __framestream Internal FFI namespace — an image the GPU drew carried out to another program, frame after frame.

Purpose

Wrap the raw __framestream.* FFI namespace in a typed Luau table that gets auto-injected as _G.frameStream via the prelude. A session binds one texture to one stream: make a render target the size you want to emit, point a Camera at it, open a stream to whoever is consuming the pixels, and attach the two. From then on the engine reads that texture back every time the session is due a frame, packs the pixels, and writes each frame to the stream. The image sits in GPU memory, where nothing outside the renderer can reach it; the session is what puts it on the wire.

The readback runs through a ring of staging buffers allocated once and reused for the life of the session, so the frame the GPU is finishing is read back while the next one renders and nothing waits on the render thread. The write refuses rather than waits, so a consumer reading slower than the engine renders falls behind by frames — visible on status().droppedBackpressure — instead of holding the renderer back.

Usage

-- A 128x32 render target, and a Camera that draws into it.
local rt = renderer.texture.create({ width = 128, height = 32 })
entity.spawn({
    name = "panelCam",
    position = { 0, 0, 6 },
    components = {
        Camera = { textureHandle = rt.guid, postProcessing = false },
    },
})

-- Somewhere to send the pixels.
local out = task.await(stream.open("tcp://127.0.0.1:9000"))

-- Carry the camera's frames out at 30 fps.
local session = frameStream.attach(rt.guid, out, { fps = 30 })

-- Later.
local s = frameStream.status(session)
print(s.frames, s.dropped, s.achievedFps)

frameStream.detach(session)
stream.close(out)

What reaches the stream

One frame's pixels, then the next frame's, with nothing between them. A frame is width * height * bytesPerPixel bytes, rows top-first, and is written in a single call — so a consumer that has counted its bytes reads a whole frame or none of it, never half of one.

format picks the layout: "rgb24" (3 bytes per pixel, the default) or "rgba8" (4) — a call naming any other format raises, naming both. flipY = true writes the last texture row first, for a consumer whose display is scanned bottom-up.

A consumer at the far end of a byte stream that can drop out and be reopened usually wants more than the pixels — a magic to resynchronise on, a length, a sequence number. That belongs to the consumer's own protocol, so it is written by whoever speaks it rather than baked in here.

Framing the bytes yourself

To put a header of your own ahead of each frame, run the frames through a loopback stream and write them on from there. The session fills the loopback with whole frames; you drain one, prepend whatever your consumer expects, and write the result to the real stream:

local FRAME = 128 * 32 * 3

local pixels = task.await(stream.open("loopback://panel", {
    inboundCapacity = FRAME * 4,
    outboundCapacity = FRAME * 4,
}))
local out = task.await(stream.open("tcp://127.0.0.1:9000"))
local session = frameStream.attach(rt.guid, pixels, { fps = 30 })

local counter = 0
task.spawn(function()
    while true do
        local frame = stream.read(pixels, FRAME)
        if #frame == FRAME then
            stream.write(out, myHeader(counter) .. frame)
            counter += 1
        end
        task.wait()
    end
end)

That costs one in-memory copy per frame and keeps the protocol in content, where it can change without an engine build. Attaching straight to the destination stream skips the copy, and is the path to take when the consumer wants the pixels bare.

Rate

fps caps how often a frame is taken and is independent of the rate the engine renders at: a session at fps = 30 in an engine running at 144 takes every fifth frame or so, and one at fps = 30 in an engine running at 20 takes every frame it can. status().achievedFps is what it reached — compare it against requestedFps to tell a display that is running slow from one that is running as asked.

Reading the status

frameStream.status() separates the three ways a frame fails to arrive, because they call for different fixes:

• droppedBackpressure — the consumer is reading slower than the session is producing. Either the consumer is too slow, or the stream's outboundCapacity is too small to absorb its jitter.
• stalledReadbacks — the frame came due while every staging buffer still held a copy on its way from the GPU. The session is asking for frames faster than the GPU is handing them back.
• lastOutcome == "tooLarge" — a frame is bigger than the stream's whole outbound capacity, so every frame of that size will be refused too. Open the stream with an outboundCapacity that holds one.

Which frame a stream holds

A frame is copied when it comes due and reaches the stream once its copy has come back from the GPU, so how many frames a stream holds at a given moment depends on how fast those copies come back. Two sessions on one texture can therefore hold different numbers of frames when they are read, the newest frame of one showing a later image than the newest of the other. status().lastCopyFrame names the engine frame, in time.frameCount() numbering, that the copy of the newest accepted frame was taken on. Every copy one engine frame takes of a texture holds the same image, so read it in the same step as draining the stream, and two sessions whose readings agree hold the same image as their last frame. Two different readings can still name one image: an engine frame on which nothing drew into the texture copies the image it already held.

Platforms

Frame egress runs on native and wasm alike: the readback is submitted and polled without blocking on either, and the packing is plain byte work. What differs between them is which transports a stream can be opened over — stream.transports() is the probe for that, and tcp names its wasm absence there.

Exports

• frameStream.attach(texture, stream, opts?) -> string?, string? — session handle, or nil + reason.
• frameStream.detach(handle) -> boolean
• frameStream.status(handle) -> FrameStreamStatus? — nil when handle names no live session.
• frameStream.list() -> { string }