Idea
Record the individual per-participant inputs to the server mixer as synchronized stems, before they are summed into each recipient's stereo mix.
The server already sees all participant blocks in one mixer step and assigns the result a common sample-based server timeline. Capturing at this point avoids trying to synchronize independent client wall clocks or recording start times.
Why this is useful
- Reconstruct a jam exactly from the audio that reached the server.
- Compare alternative offline mixes, including ordinary stereo panning, HRTF binaural rendering, and virtual-room processing.
- Export multitrack sessions for rehearsal review or later production.
- Use each server stem as a synchronized guide track for a higher-quality local recording uploaded after the session.
- Align a local recording by waveform correlation, estimate its audio-interface clock drift, warp it onto the server timeline, and replace the network stem.
- Repair network dropouts when the corresponding local recording is available.
Possible recording format
Use one segmented file per participant/stream rather than a single large multichannel file, plus a session manifest containing at least:
- session and participant/stream identity;
- channel names and layouts;
- filename and segment start/end positions in server samples;
- source packet/message counters where available;
- joins, leaves, channel-layout changes, missing blocks, and other discontinuities;
- sample rate and recording format.
For a missing input block, write silence to preserve the shared timeline and add a gap entry to the manifest.
Implementation constraints
- Do no file I/O on MixerThread.
- Copy/enqueue incoming blocks into a bounded recording queue before ServerMixerCore::bufferMixdown().
- Let a lower-priority worker own file creation, writing, finalization, and manifest updates.
- Recording overload must drop or mark recording data rather than interfere with live audio mixing.
- Preserve stereo or multichannel participant inputs without prematurely summing them.
- Segment cleanly when participants reconnect or their physical channel count changes.
Follow-up: double-ended polishing
A later client-side sidecar could map local recording sample positions to outgoing packet counters. Combined with the server manifest's packet-counter-to-server-sample mapping, this would make replacement with pristine local recordings deterministic. Waveform correlation should remain a validation and fallback mechanism, and can also estimate residual clock drift.
This feature does not require a globally synchronized wall clock. It can use the existing per-session server sample timeline and can complement the authoritative transport work in #69 later.
Initial completion criterion
A recorded test session can be imported into a DAW with every participant on isolated, sample-aligned tracks, and an offline recreation using the recorded stems nulls against or closely matches the server's live mix apart from explicitly documented recording gaps or format quantization.
Idea
Record the individual per-participant inputs to the server mixer as synchronized stems, before they are summed into each recipient's stereo mix.
The server already sees all participant blocks in one mixer step and assigns the result a common sample-based server timeline. Capturing at this point avoids trying to synchronize independent client wall clocks or recording start times.
Why this is useful
Possible recording format
Use one segmented file per participant/stream rather than a single large multichannel file, plus a session manifest containing at least:
For a missing input block, write silence to preserve the shared timeline and add a gap entry to the manifest.
Implementation constraints
Follow-up: double-ended polishing
A later client-side sidecar could map local recording sample positions to outgoing packet counters. Combined with the server manifest's packet-counter-to-server-sample mapping, this would make replacement with pristine local recordings deterministic. Waveform correlation should remain a validation and fallback mechanism, and can also estimate residual clock drift.
This feature does not require a globally synchronized wall clock. It can use the existing per-session server sample timeline and can complement the authoritative transport work in #69 later.
Initial completion criterion
A recorded test session can be imported into a DAW with every participant on isolated, sample-aligned tracks, and an offline recreation using the recorded stems nulls against or closely matches the server's live mix apart from explicitly documented recording gaps or format quantization.