Programmatic 2D & 3D Video Motion Engine for AI Agents and Developers
-yellow.svg){kind=icon}
Named after the popular crispy Filipino spring roll, Lumpia is a high-performance, single-binary video motion engine written in Golang. It renders hardware-accelerated 2D vector typography and 3D scenes via Raylib, evaluates embedded ES6 JavaScript via Goja, and streams raw RGBA frames directly into FFmpeg via stdin pipes.
Unlike traditional Node.js/Puppeteer video rendering tools, Lumpia requires zero web toolchains, zero npm dependencies, zero tsconfig setup, and no headless Chrome binaries.
Lumpia is not just a frame-by-frame video renderer. Because it runs on Raylib's native OpenGL game loop + Go's high-performance concurrency, it can do what web-based tools like Remotion or Motion Canvas cannot: real-time interactive 60 FPS gameplay and live streaming overlays.
Scenes can become interactive game loops driven by keyboard and mouse input:
runFrame(ctx) {
const { canvas, input, state } = ctx;
// Move player with arrow keys
if (input.isKeyDown("ArrowRight")) state.playerX += 5;
if (input.isKeyPressed("Space")) state.jump();
canvas.drawCircle(state.playerX, state.playerY, 20, "#22c55e");
}- Live Recording: Play in real-time at 60 FPS while Lumpia streams frames directly into FFmpeg.
- Input-Log Replay (Deterministic): Log input events per tick, then replay deterministically for zero-dropped-frame 4K output with perfect audio sync — even if your hardware lagged during gameplay.
Ideal for: Short-form content (TikTok/Reels), generative game replays, interactive quizzes, and motion comics.
Lumpia can run as a real-time motion graphics server for OBS, Twitch, or YouTube:
({
config: { width: 1920, height: 1080, fps: 60, stream: true },
onEvent(event, state) {
if (event.type === "DONATION") {
state.alertText = `${event.user} donated $${event.amount}!`;
state.alertStartTime = ctx.time;
}
},
runFrame(ctx) {
const { canvas, time, state } = ctx;
// Render dynamic overlays, lower thirds, alerts...
}
})- Transparent Output: Native transparent/borderless window via Raylib — capture directly in OBS via Game Capture with alpha.
- WebSocket API: Trigger animations from external sources (Twitch events, Streamlabs, mobile dashboard).
- NDI / Spout2: Zero-latency shared texture output into OBS/vMix.
Ideal for: Stream startingsoon/BRB screens, live alerts, data-driven tickers, and real-time audio visualizers.
| Feature | Remotion / Motion Canvas | Lumpia (Go + Raylib) |
|---|---|---|
| Engine Runtime | Headless Chrome / Node.js | Native C (OpenGL) via Go |
| Real-time 60 FPS Interactive | Laggy in browser | Native Game Loop |
| RAM / CPU Overhead | 500MB+ per overlay | ~30-80MB RAM |
| Stream Overlays | Heavy OBS Browser Sources | Native Transparent Window + NDI |
- 🚀 Single Binary: Everything compiles into a standalone Go executable with zero external DLLs or shared libraries.
- 🚫 Zero Node/npm Toolchain: Embedded pure-Go JavaScript engine (Goja). No
node_modules,esbuild, or web bundlers. - 🎨 Native 2D + 3D Canvas: Built on Raylib, supporting vector shapes, anti-aliased text, 3D cameras, lights, and 3D primitives out of the box.
- 🌀 Analytical Spring Physics: Closed-form differential spring equation solver (
lumpia.math.spring) computes exact motion at any timestamp$t$ without frame-stepping history. - 🎬 Headless FFmpeg Streaming: Streams raw RGBA pixel byte slices from GPU render targets directly into FFmpeg via
os/execstandard input. - 🤖 AI Agent Native: Designed for LLM function-calling, prompt-driven motion graphics generation, and multimodal vision feedback loops.
┌───────────────────────────┐
│ scene.js (ES6 JS) │
└─────────────┬─────────────┘
│
▼
┌───────────────────────────┐
│ Goja JS Engine │
│ (100% Pure Go VM) │
└─────────────┬─────────────┘
│
┌─────────────────────┴─────────────────────┐
│ │
▼ ▼
┌──────────────────────┐ ┌──────────────────────┐
│ raylib-go │ │ Analytical Springs │
│ (2D/3D Rendering) │ │ Native Go Math │
└──────────┬───────────┘ └──────────────────────┘
│ Offscreen Pixels ([]byte)
▼
┌──────────────────────┐
│ os/exec FFmpeg Pipe │
│ (Write RGBA Bytes) │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ output.mp4 │
└──────────────────────┘
- Go 1.20 or higher
- FFmpeg installed and accessible in your system
PATH
Raylib requires standard C/OpenGL header packages:
sudo apt update
sudo apt install -y build-essential libgl1-mesa-dev libxi-dev libxcursor-dev libxrandr-dev libxinerama-dev libwayland-dev libxkbcommon-devbrew install ffmpegInstall FFmpeg and add it to your Windows System Path.
# Clone repository
git clone https://github.com/your-username/lumpia.git
cd lumpia
# Download Go dependencies & build binary
go mod tidy
go build -o lumpia main.go./lumpia render examples/scene.js -o output.mp4Scenes in Lumpia are plain ES6 JavaScript files that export a configuration object and a runFrame(ctx) lifecycle function.
// examples/scene.js
({
config: {
width: 1280,
height: 720,
fps: 30,
duration: 5.0 // seconds
},
runFrame(ctx) {
const { canvas, time, width, height } = ctx;
// 1. Clear background with hex color
canvas.drawRect(0, 0, width, height, "#0a0a0f");
// 2. Render Orbiting 3D Cube Scene
canvas.begin3D();
const rx = Math.sin(time * 2.0) * 5.0;
const rz = Math.cos(time * 2.0) * 5.0;
canvas.drawCube(rx, 0, rz, 3.0, 3.0, 3.0, "#22d3ee");
canvas.end3D();
// 3. Compute Analytical Spring Scale for Typography
const scale = lumpia.math.spring({
from: 0,
to: 1,
time: time,
stiffness: 220,
damping: 12
});
// 4. Render 2D Motion Graphic Text
canvas.drawText("LUMPIA GO ENGINE", 100, 100, Math.floor(40 * scale), "#ffffff");
}
})Every call to runFrame(ctx) receives the current frame context:
| Property | Type | Description |
|---|---|---|
ctx.time |
number |
Elapsed time in seconds (tick / fps). |
ctx.localTime |
number |
Progress from 0.0 (start) to 1.0 (end). |
ctx.tick |
number |
Current frame index (0, 1, 2...). |
ctx.width |
number |
Render width in pixels. |
ctx.height |
number |
Render height in pixels. |
ctx.canvas |
object |
Drawing context (methods below). |
// Draw filled rectangle
ctx.canvas.drawRect(x, y, width, height, hexColor);
// Draw filled circle
ctx.canvas.drawCircle(centerX, centerY, radius, hexColor);
// Draw text
ctx.canvas.drawText(text, x, y, fontSize, hexColor);// Enter 3D perspective camera mode
ctx.canvas.begin3D();
// Draw 3D cube with wireframe outline
ctx.canvas.drawCube(posX, posY, posZ, width, height, length, hexColor);
// Exit 3D perspective mode
ctx.canvas.end3D();Analytical closed-form spring solver. Solves underdamped, critically damped, and overdamped differential equations instantly for any timestamp
const value = lumpia.math.spring({
from: 0.0, // Starting value (default: 0)
to: 1.0, // Target value (default: 1)
time: ctx.time, // Elapsed time in seconds
stiffness: 180, // Spring stiffness constant k (default: 180)
damping: 12, // Damping factor c (default: 12)
mass: 1.0 // Mass m (default: 1.0)
});Lumpia is built for AI Agent orchestration (GPT-4o, Claude 3.5, Gemini 1.5):
- Schema Extraction: Agents fetch blueprint tool definitions via Lumpia CLI.
- Scene Code Generation: Agents generate clean ES6
scene.jsfiles containing layouts and spring motion parameters. - Execution: Lumpia renders frames headlessly in Go.
- Multimodal Feedback: Vision models inspect keyframe PNG snapshots generated by Lumpia to evaluate layout, contrast, and visual timing.
lumpia/
├── main.go # CLI command router
├── pkg/
│ ├── engine/
│ │ ├── engine.go # Raylib window & render texture manager
│ │ └── encoder.go # FFmpeg pipe stdin writer
│ ├── js/
│ │ └── host.go # Goja VM host & canvas API bindings
│ └── math/
│ └── spring.go # Analytical spring solver
└── examples/
└── scene.js # Sample ES6 motion scene
This project is open-source software licensed under the MIT License.