I'm a Chinese speaker living in the US. Weekdays are mostly English: standup on Zoom, design docs, Slack threads that somehow always explode at 5pm. Nights and weekends are Japanese — VTuber streams, cooking livestreams, the occasional anime episode I refuse to watch with English-only subs.
For months my "language practice" looked like this: Twitch / YouTube open in Chrome Host says something fast I miss it Pause Scrub back 8–12 seconds Still not sure → open a dictionary tab Stream moves on without me
That loop is survivable for a recorded course. It is miserable for a live stream where chat is already three jokes ahead.
I didn't want another upload-a-video translator. I wanted: stay on the same tab, see the original line + a Chinese (or English) translation almost immediately, keep watching.
So I built a Chrome/Edge extension that captions whatever audio is playing in the tab, and put the realtime path on Cloudflare Workers + Durable Objects.
Hard requirements from day one: Works on any tab with audio — YouTube, Twitch, Netflix-in-browser, Zoom/Meet in the browser, random course sites No file upload Bilingual overlay (source + target) Latency low enough that a live stream still feels watchable (~0.5s ballpark for me) Prefer keeping raw audio in the tab; only ship text / small chunks for processing
That ruled out "export MP4 → wait → download SRT." The extension had to own capture; the edge had to own the session.
Extension (Chrome MV3) Uses tabCapture / offscreen document patterns to grab the tab's audio stream, run VAD-ish chunking so I'm not uploading silence, and render a floating bilingual overlay. Settings and history live locally first.
Worker (stateless edge) Auth, plan checks, routing. A request comes in: "here's a short audio/text segment for session X." The Worker doesn't hold long-lived state — it just verifies the user and hands the work to the right place.
Durable Object (one per caption session) This is the part that made the architecture click for me.
Each Start click maps to a Durable Object keyed by session id. That object: Owns the WebSocket to the extension (hibernation-friendly) Queues / serializes ASR + translation steps so chunks don't arrive out of order on screen Tracks minute usage for the free tier without racing a shared Redis Keeps a short rolling buffer of recent caption lines (enough to recover a reconnect without rebuilding history from scratch) Tears down cleanly when the user hits Stop or the tab dies
I tried the "just shove everything through a single Worker + external WebSocket server" version first. It worked until two tabs and a flaky Wi‑Fi night. Per-session Durable Objects made the failure domain obvious: one stuck session doesn't take down everyone else's captions.
Why Cloudflare specifically I didn't want to babysit a caption server in us-west-2 while I'm watching a stream at 1am PT. Workers put the handshake close to me; Durable Objects give me single-threaded session semantics without standing up Redis + a sticky-session LB for a side project that started as "please stop pausing."
ASR / translation providers sit behind the Worker as normal HTTPS calls. The interesting bit for me wasn't swapping models — it was keeping the session glue boring and correct at the edge.
I didn't optimize for leaderboard numbers. I optimized for: can I leave this on for a 90-minute cooking stream without rage-pausing?
For me, captions landing around half a second after speech is the line between "tool" and "distraction." Faster is nicer; much slower and my thumb goes back to the spacebar.
Typical flow now: Install the extension Open the Japanese stream (or the English Zoom tab for work) Pick target language — Chinese when I'm tired, English when I'm taking notes Hit Start Read along: original on top, translation under it
Meetings: browser Meet/Zoom tab → Start → stop pretending I caught every acronym. Language nights: dual subtitles on, dictionary tab closed unless I really want a deep dive later.