Skip to content

feat(producer): distributed parallel chunk rendering#658

Closed
miguel-heygen wants to merge 1 commit into
mainfrom
feat/distributed-chunk-rendering
Closed

feat(producer): distributed parallel chunk rendering#658
miguel-heygen wants to merge 1 commit into
mainfrom
feat/distributed-chunk-rendering

Conversation

@miguel-heygen
Copy link
Copy Markdown
Collaborator

Summary

Implements the "big lever" from the cloud rendering research: distributed parallel chunk rendering via Temporal workflows. Instead of rendering a full composition on one machine, split the timeline into N chunks and render them on separate workers in parallel.

Current state: HyperFrames already has local parallel rendering (--workers N) that runs multiple Chrome instances on one machine. This PR adds the infrastructure for distributed parallel — each chunk rendered on a separate machine, coordinated by Temporal.

Architecture

Temporal Parent Workflow
  ├── Probe Activity → metadata (totalFrames, fps, dimensions)
  ├── Chunk 1 Activity → renders frames 0-249 → segment_0.mp4
  ├── Chunk 2 Activity → renders frames 250-499 → segment_1.mp4
  ├── Chunk 3 Activity → renders frames 500-749 → segment_2.mp4
  ├── Chunk 4 Activity → renders frames 750-899 → segment_3.mp4
  └── Stitch Activity → concat segments + mux audio → final.mp4

Changes

Phase 1: Producer API (this commit)

  • startFrame/endFrame/skipAudio fields on render request
  • frameRange/skipAudio on RenderConfig — clamped to composition bounds
  • POST /probe endpoint — compiles composition, returns metadata without rendering

Phase 2-5: Temporal (TODO)

  • Pydantic models for probe, chunk render, stitch activities
  • Three new activities: probe, chunk render, stitch
  • HyperframeRenderParallelWorkflow parent orchestrator
  • Registration in worker configs

Expected Impact

  • 30s video with 4 chunks: ~15s wall-clock (vs ~60s sequential)
  • 1-min video with 8 chunks: ~20s (vs ~120s)
  • Scales linearly with available producer pods

Test plan

  • TypeScript typechecks pass
  • Lefthook pre-commit passes (lint + format + typecheck)
  • Unit tests for frame range clamping
  • Integration test: render with startFrame/endFrame produces correct subset
  • Temporal workflow E2E test

🤖 Generated with Claude Code

…stributed chunk rendering

Phase 1 of distributed parallel rendering:

- RenderInput: add startFrame/endFrame/skipAudio fields
- RenderConfig: add frameRange and skipAudio options
- /probe endpoint: returns composition metadata without rendering
- Both /render and /render/stream forward the new fields

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@miguel-heygen
Copy link
Copy Markdown
Collaborator Author

Moving to internal repo.

@miguel-heygen miguel-heygen deleted the feat/distributed-chunk-rendering branch May 7, 2026 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant