Skip to content

Audio caching policy: cache music + plain speech, NOT segment-aware speech #220

Description

@SecurityQQ

Policy

Audio is cached at the render layer (render/src/api/render/cache-utils.ts), with one exception: segment-aware speech is not media-cached.

Audio kind Media-cached? Why
Music (generateMusic) ✅ yes no timing metadata to lose
Plain speech (no segments) ✅ yes no timing metadata to lose
Segment-aware speech (await Speech({ children: [...] })) ❌ no carries segments/words/duration a media blob can't hold

Segment-aware speech is still cached losslessly via the SDK's own resolveSpeech cache key (sdk/src/react/resolve.ts), which preserves the full CachedSpeechResult.

How it's enforced

  • extractMediaData returns null for anything shaped like CachedSpeechResult ({ file, duration, words?, segments? }) → not written to the media cache.
  • extractMediaData extracts { audio: {...} } payloads (music + plain generations) → written to the media cache.
  • reconstructCachedValue returns audio in the { audio: {...} } shape the SDK destructures (const { audio } = ...).

History (why this exists)

  1. PR chore: remove fal-timing debug prints #91 media-cached all audio → dropped segments → crashed segments[0].duration.
  2. Revert feat(fal): persist queue request IDs to recover from timeouts #93 (f2e7ee9) removed the { audio } wrapper but left stale entries → crashed audio.mediaType.
  3. Current fix: cache audio except segment-aware speech, and restore the { audio } wrapper.

Related: vargHQ/render#97, #219.

If you touch this

Never media-cache a value that has segments/words/duration — it must round-trip through the SDK resolveSpeech cache instead.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions