That definition is the easy part. Every explainer on the internet will tell you a voice agent is speech-to-text, plus a language model, plus text-to-speech.

Here is the part almost nobody writes down: a voice agent is a latency and connectivity problem, not a prompt problem.

The language model is the easy 80%. Wiring three models together is a weekend project. The hard part — the part that decides whether your agent feels human or just goes silent in a customer's office — is the plumbing underneath: how the audio actually travels, how fast, and whether it can connect at all.

This guide covers the whole stack. You'll get the standard architecture, the honest 2026 trade-offs, a real latency budget with numbers we measured ourselves, and the transport layer that the top-ranking guides leave out entirely.

TL;DR: An AI voice agent hears you (speech-to-text), thinks (an LLM), and speaks back (text-to-speech), coordinated by an orchestrator that manages turns and interruptions — with every stage streamed to stay under the ~800 ms that keeps a conversation feeling natural. The layer most guides skip is transport: the audio rides WebRTC, and on real-world networks it needs a TURN relay, or the agent connects and goes silent.

What Is an AI Voice Agent?

An AI voice agent is an autonomous, voice-first system that holds a natural spoken conversation, reasons about it in real time with a large language model, and takes action across connected systems — without a human scripting each turn. It combines speech-to-text, an LLM, and text-to-speech, connected over a real-time media transport like WebRTC.

That last clause is the one you rarely see, and it is the whole reason this article exists.

The difference between a voice agent and the older systems it replaces is intent. An IVR forces callers down a fixed menu tree — "press 1 for sales." A chatbot handles typed text. A voice agent understands natural spoken language, decides what to do, calls your backend, and answers in speech, per the distinction drawn across Aircall's 2026 explainer and Deepgram's 2026 guide.

So a voice agent is a loop, not a black box. It hears, it thinks, it speaks, and something has to conduct all three in the right order, fast enough that you don't notice the seams.

That conductor is the orchestrator, and it does more than pass data between stages. Let's break the loop down stage by stage.

How Do Voice Agents Work? The Four Stages

A voice agent runs four real-time stages in a tight loop: speech-to-text transcribes what you say, an LLM decides the reply and calls any tools, text-to-speech speaks it, and an orchestrator manages when a turn ends and when to stop for an interruption. Streaming overlaps the stages so the reply starts before the model has finished thinking.

Here's each stage and what it's actually responsible for.

Speech-to-text (STT) turns your audio into words the model can read. In a good agent it runs continuously, emitting partial transcripts as you talk rather than waiting for you to finish.

The LLM reads the transcript, decides what to say, and — when needed — calls tools: look up an order, book the appointment, check inventory. This is the "brain," but as you'll see in the latency section, it is rarely the slow part.

Text-to-speech (TTS) turns the reply back into audio. Streaming TTS starts speaking the first words while the rest of the sentence is still being generated, which is what keeps the pause short.

The orchestrator is the unsung hero. It decides when your turn has ended (turn detection), starts the reply, and — critically — stops the agent mid-sentence the instant you interrupt. That interruption behavior is called barge-in, and getting it right is most of what makes an agent feel human rather than robotic.

Notice the theme: everything streams. A voice agent that waits for each stage to fully finish before starting the next one feels broken, because the silence stacks up.

AssemblyAI's April 2026 architecture breakdown puts a number on it: a naive, non-streaming pipeline adds two to four seconds of dead air per turn. Nobody waits four seconds for a "hello."

That's the standard model. But in 2026 there's a real architectural fork in the road — one model or three?

Cascaded Pipeline vs Speech-to-Speech

There are two ways to build the "hear-think-speak" loop, and the choice is a genuine engineering trade-off in 2026. A cascaded pipeline chains three separate models — STT, then LLM, then TTS — with readable text between each step. A speech-to-speech (S2S) model does it in one shot: audio in, audio out, no text in the middle.

Cascade gives you a text artifact at every step, so you can log it, moderate it, filter it, and route on it. Speech-to-speech often feels more natural and can be faster, but it's harder to debug, more expensive, and less transparent.

As of April 2026, cascade still dominates production, per Deepgram and Softcery's lab tests. Here is how the two stack up on the numbers people actually argue about.

The cost gap is not a rounding error. Softcery's April 2026 measurements show a spread of up to 182× between the cheapest cascaded stack and premium realtime speech-to-speech models — the difference between a fraction of a cent and roughly thirty cents a minute.

Cascade runs about $0.05–$0.15 per minute and produces a clean text log at every step; premium speech-to-speech models run roughly $0.15–$0.60 per minute and feel more natural but are far harder to audit. Their time-to-first-audio ranges from about 0.78 seconds on the fastest model to nearly 3 seconds on others, versus roughly 1.5–3 seconds end-to-end for a well-built cascade.

Here's what most teams actually do: a hybrid. Use a speech-to-speech model for the natural-feeling opener, then fall back to a cascaded pipeline for the transactional turns where you need a text log and tool calls.

Now, every source above measures the models. None of them measures the wire the audio travels on. That's the layer we're built to explain.

The Layer Most Guides Skip: How the Audio Actually Travels

Here is the gap. The best voice-agent architecture guides — AssemblyAI's (April 2026) and Deepgram's (2026) among them — walk through STT, LLM, and TTS in detail and then stop. Deepgram names WebRTC as a transport and moves on. None of them explains the layer underneath: how the audio actually gets from a user's microphone to your agent and back.

So let's fill it in.

Audio has to ride something. There are three real choices for moving real-time voice, and they are not interchangeable.

WebRTC is the transport built for exactly this job. It runs over UDP, was designed for low-latency real-time media, handles packet loss gracefully, and includes the machinery to punch through firewalls. The emerging consensus for production voice agents is WebRTC over WebSockets.

WebSockets run over TCP. They're perfect for control messages, transcripts, and prototypes — but TCP's head-of-line blocking means one late packet stalls everything behind it, which is exactly the wrong property for live audio. Fine for a demo; not what production settles on.

SIP is the telephony path. If your agent answers actual phone calls over the PSTN, SIP is in the picture — it's a live transport decision for phone-based agents, as relinns' 2026 comparison lays out. For a web or app-based agent, WebRTC is the default.

So for the agents most people are building — web and app — the answer is WebRTC. But choosing WebRTC is where the connectivity problem begins, not where it ends.

Because WebRTC prefers a direct UDP path, and on a lot of real-world networks, that direct path simply doesn't exist. Hold that thought — we'll get to why in two sections. First, let's put real numbers on "fast enough."

The Real Latency Budget (With Our Measured Numbers)

Here's the target: keep the end-to-end response under about 800 milliseconds at the 95th percentile, from the moment you stop talking to the moment the agent starts. Under 500–700 ms feels natural; past that, it starts to feel like a bad phone connection.

Those thresholds come from practitioners. AssemblyAI's April 2026 breakdown budgets 600–900 ms for a fully streamed pipeline; Prodinit's 2026 production guide puts the reliability floor at sub-800 ms p95 and calls sub-250 ms p50 achievable. AssemblyAI also notes that responses beyond 500–700 ms "start to feel unnatural."

Here's a dated budget, stage by stage, from the published sources next to our own measured build.

Let me be honest about our numbers. We built a real voice agent and measured it on 2026-07-14 — a MacBook Pro, 16 GB RAM, on a residential network, running a cloud stack of OpenAI GPT plus Whisper STT plus OpenAI TTS with streaming enabled. These are our numbers on our hardware, not a vendor performance guarantee. The full build and method are in our TypeScript voice-agent tutorial.

End to end, we measured roughly 2.5–3.9 seconds to first audio on the default buffered pipeline. That's slower than the streamed-ideal industry budget — roughly 200–500 ms for STT, 150–400 ms for the LLM's first token, 200–400 ms for TTS, and 50–150 ms of network, about 600–900 ms end to end (AssemblyAI, 2026-04-29). The gap between that textbook 600–900 ms and a real 2.5–3.9 s is buffering and cold starts, not model quality — which is exactly the point.

And here's the insight that reorganizes how you optimize: STT dominates. Buffered Whisper alone ate 1.2–1.8 seconds — the single biggest slice. The LLM (0.8–1.4 s) and TTS (0.5–0.7 s) were not our bottleneck; transcription was. If you're trying to make an agent feel faster, the streaming STT model is usually the highest-leverage lever, not a bigger LLM.

One more number worth staring at: barge-in. When we talked over the agent, the server's voice-activity detection cancelled its speech within about 1 millisecond of detecting our voice. Interruption handling isn't a latency problem — it's an architecture problem, and it belongs on the server, close to the media.

Which brings us back to the connectivity thread. All of this latency assumes the audio connects in the first place. On a lot of networks, it doesn't.

Why Voice Agents Fail on Corporate, Hospital, and Mobile Networks

This is the failure mode that ships to production and blindsides teams: the agent works perfectly on your Wi-Fi and goes silent in a customer's office. The call connects, the transcript even flows — and there's no audio.

It's not a bug in your code. It's the network, and it's predictable.

WebRTC wants a direct peer-to-peer path over UDP. Restrictive networks break that in two ways. Many corporate, hospital, and mobile carrier networks use symmetric NAT, which scrambles the address mapping so the two sides can't agree on where to send packets. Others simply block the UDP ports WebRTC reaches for.

Here's the pattern we see over and over: it works on the developer's home Wi-Fi and dies the moment a real user is on a corporate LAN or on cellular. Same code, different network, silent call.

Now add the twist that makes this worse for agents specifically. A browser-to-browser call can sometimes fall back to a direct path between two consumer networks. A server-side voice agent has no peer-to-peer fallback — one end is a machine in a data center — so when the direct path fails, there is no plan B on the same wire. The audio has to be relayed, which is why agent media is almost always relayed in production. We wrote up that reasoning in depth in TURN for AI voice agents.

So the uncomfortable truth is that "it works on my machine" is the default state of a WebRTC voice agent, and it lies to you. The networks where it fails are precisely the networks your paying customers sit on.

The fix is a relay. And that's where STUN and TURN come in.

STUN, TURN, and What a Production Agent Actually Needs

Two acronyms do the connectivity work, and they are not the same thing. STUN helps a client discover its own public address so two peers can try a direct connection. TURN is the fallback that actually relays the media through a server when the direct path fails. STUN discovers; TURN relays. Confusing the two is the most common mistake in this whole topic.

For a production voice agent, you need TURN. The direct path fails often enough on real networks — and a server-side agent has no peer-to-peer fallback — that a relay isn't a nice-to-have, it's the thing standing between "works in the demo" and "works for customers."

There's a detail that matters for the hardest networks: TURN over TLS on port 443. Locked-down corporate and hospital firewalls that block everything else usually still allow outbound 443, because that's where normal HTTPS lives. A TURN server that speaks TURNS on 443 looks like ordinary web traffic and gets through where raw UDP is dead on arrival.

So how do you get a TURN server? You have two honest paths, and both are legitimate.

Run your own with an open-source server like coturn. It's free software, but you own the config, the TLS certificates, the ports, the capacity planning, and the bandwidth bill. It's real DevOps work, and the bandwidth adds up.

Use a managed TURN service and skip the operations. You can start free — Open Relay gives you 20 GB/month of TURN bandwidth at no cost, on ports 80 and 443 with TURNS, which is plenty for small workloads. When you outgrow it, Metered's TURN service starts with a 500 MB free trial and then runs Growth at $99 for 150 GB, Business at $199 for 500 GB, and Enterprise at $499 for 2 TB, across 31+ regions and 100+ edge locations.

You don't have to decide today. The point is that a TURN relay is part of the voice-agent stack, full stop — not an optional extra you bolt on after launch, but the layer that makes the other four work on real networks. If you want to check whether your own agent's path holds up, you can test a TURN connection before your users do it for you.

That's the whole stack, actually. Let's map it.

The Voice-Agent Stack You Actually Need

A voice-agent stack has six layers, and most guides only cover the first four. You need speech-to-text, an LLM, text-to-speech, an orchestrator for turn-taking and barge-in, a media-transport layer (WebRTC) to move the audio, and — for production reliability — a TURN relay plus signalling to establish the connection.

The first four are the conversation. The last two are the connection. Skip the connection layers and you get an agent that demos beautifully and fails in the field.

Metered is not a voice-agent platform, and this isn't a pitch to replace your STT, LLM, or TTS. It is the infrastructure under the stack — the transport and connectivity layer that sits beneath any voice agent, whether you built it yourself or bought a hosted one.

That layer is two things. TURN relays the media so the agent connects on real networks; Open Relay covers the free tier and the Metered TURN service covers scale, regions, and analytics. And signalling is the coordination channel that helps the two sides find each other and exchange connection details before the media flows.

On signalling, one honest note so you can plan the whole connection layer: Metered Realtime is a managed signalling service you can start free, with an MIT-licensed open-source client.

Read the stack top to bottom and the thesis of this whole guide falls out. The top four layers are where the intelligence lives, and they're mostly solved by picking good models. The bottom two are where agents actually break, and they're an infrastructure problem — latency and connectivity — not a prompt problem.

So how do you assemble all six? You have three routes.

How to Build a Voice Agent

build it open-source.** Assemble it yourself for full control and provider freedom. Our own free, open-source TypeScript SDK, LLMRTC, handles the voice-agent hard parts — WebRTC transport, server-side voice-activity detection, natural barge-in, tool calling, and a provider-agnostic pipeline so you can swap OpenAI, Anthropic, Gemini, or local models by config. It's Apache-2.0 and truly free, built by a team that runs production WebRTC infrastructure. Its own docs tell you to put a TURN server in front of it for users behind NAT — because, as this article has hammered, you need one.

We proved that route end-to-end: our TypeScript voice-agent tutorial builds a real agent and swaps the entire stack from cloud to 100% local with one config change. And if you're wiring a browser straight to a hosted realtime model, our walkthrough of OpenAI Realtime over WebRTC shows exactly when TURN enters the picture — the direct browser-to-OpenAI path connected with no iceServers at all, and TURN came back the moment we owned a leg of the connection.

Route 2: hybrid. Use a platform or a hosted model for the conversation, and own the connectivity layer yourself — WebRTC transport, TURN relay, signalling — so you control quality, regions, and cost on the part that actually breaks. This is where most serious deployments land, and it's the route where an independent TURN service earns its keep.

Whichever route you take, the connection layer is yours to get right. No platform makes the network problem disappear if you own any leg of the WebRTC path.

Frequently Asked Questions

What is an AI voice agent?

An AI voice agent is an autonomous, voice-first system that holds a natural spoken conversation, reasons about it in real time with a large language model, and takes action across connected systems — without a human scripting each turn. It combines speech-to-text, an LLM, and text-to-speech, connected over a real-time media transport like WebRTC.

How do AI voice agents work?

They run four real-time stages: speech-to-text transcribes the user, an LLM decides the response and calls tools, text-to-speech speaks it, and an orchestrator manages turn-taking and interruptions. Streaming overlaps the stages to cut latency. A media-transport layer — usually WebRTC — carries the audio between the user and the agent.

How much latency is acceptable for a voice agent?

Keep end-to-end response under about 800 ms at p95, from end of speech to first audio; under 500–700 ms feels natural, and sub-250 ms p50 is achievable with a fully streamed stack. A naive, non-streaming pipeline adds 2–4 seconds of dead air, which breaks the conversation. Transport is part of that budget.

Do voice agents need a TURN server?

In production, usually yes. A server-side agent's WebRTC media has no peer-to-peer fallback, and many corporate and mobile networks block direct UDP, so the audio must be relayed through a TURN server. You can start free with Open Relay's 20 GB/month and move to a managed TURN service for regions, capacity, and analytics as you scale.

Why does my voice agent work locally but fail on office or hospital Wi-Fi?

Restrictive networks use symmetric NAT and block the UDP ports WebRTC needs, so the direct media path can't form. A server-side agent has no peer-to-peer fallback, so its audio is almost always relayed — which means it needs a TURN server. Without one, the call connects and then goes silent.

What's the difference between speech-to-speech and a cascaded pipeline?

A cascaded pipeline chains three models — STT, LLM, TTS — giving you a readable text artifact at each step to log, filter, or route. Speech-to-speech uses one model from audio in to audio out: often more natural, but harder to debug, more expensive, and less transparent. As of April 2026, cascade still dominates production.

What's the difference between a voice agent and an IVR or chatbot?

An IVR forces callers down fixed menus ("press 1 for sales"); a chatbot handles typed text. A voice agent understands natural spoken language, reasons with an LLM, takes actions in backend systems, and replies in natural speech — no scripted menu tree, and it works over the phone or the web.

The Bottom Line

An AI voice agent is four models in a loop — hear, think, speak, and a conductor to run them — riding on two layers of connectivity most guides never mention. Get the models right and you have a demo. Get the transport and relay right and you have a product.

That's the whole argument: voice agents are a latency and connectivity problem, not a prompt problem. The prompt is the part you'll finish first. The media path is the part that decides whether your agent feels human or goes silent on the exact networks your customers use.

So build the conversation however you like — a platform, open-source, or a hybrid. But own the connection layer. Start free on Open Relay's 20 GB/month, and when real users behind real firewalls show up, Metered's TURN service relays the audio across 31+ regions so your agent connects everywhere — not just on your Wi-Fi.

About the author: This guide was written by James Bordane

Do you need a TURN server to use the OpenAI Realtime API over WebRTC? It is the first question a WebRTC-experienced developer asks, and the answer is stranger than yes or no: on the direct browser-to-OpenAI path you do not — and you could not add your own TURN even if you wanted to.

But the moment you build the architecture most production voice agents actually ship, TURN comes back onto the critical path.

We map the real connection topologies, walk the WebRTC handshake with a complete working example we executed against the live API, and give you a decision table for exactly when a TURN server is mandatory versus irrelevant.

TL;DR: The OpenAI Realtime API runs over WebRTC, WebSocket, or SIP (OpenAI docs, 2026-07-17). On the direct browser-to-OpenAI WebRTC path you do not need your TURN server: OpenAI uses public endpoints, host candidates only, and a TCP/443 fallback, so you can't add TURN anyway. The moment you own a WebRTC leg (browser to your server, or a Python aiortc agent), TURN over TLS on 443 is mandatory on restrictive networks.

Does the OpenAI Realtime API need a TURN server?

No — not on the direct path, and yes — the instant you add your own server.

If your browser talks straight to OpenAI, OpenAI owns the WebRTC endpoint. It publishes public, reachable addresses and handles restrictive networks with its own TCP/443 fallback. There is no TURN server for you to configure.

If you put your own server in the middle — to hold your API key, add tools and guardrails, record calls, or swap providers — you now own a browser-to-your-server WebRTC connection. That leg needs your STUN and TURN, exactly like any other WebRTC app.

Most real production voice agents are the second case. So the honest answer is "usually yes — but probably not for the reason you'd expect, and not on the leg you'd expect."

What the OpenAI Realtime API is in 2026

The Realtime API is OpenAI's low-latency, speech-to-speech interface for building a realtime voice AI agent. It went generally available on 2025-08-28 alongside the first production model, gpt-realtime (OpenAI, accessed 2026-07-17).

As of 2026-07-17, the catalog lists gpt-realtime-2 as the default realtime model, with gpt-realtime-2.1 and a cheaper gpt-realtime-2.1-mini as the current point releases, plus specialized gpt-realtime-translate and gpt-realtime-whisper models (developers.openai.com, accessed 2026-07-17). OpenAI ships these fast — four point releases in about eleven months — so pin a specific model and date it rather than trusting "the latest."

The API is reachable over three transports, and OpenAI gives explicit guidance on each. This choice decides which leg of your system owns NAT traversal, so read the table with that lens.

Table: OpenAI Realtime API transports and guidance, quoted from the official Realtime guide (developers.openai.com, accessed 2026-07-17).

The load-bearing takeaway: WebRTC is the client-edge transport, WebSocket is the server-side transport. Where your audio originates tells you which transport to use — and whether a browser is a WebRTC peer at all.

How the browser-to-OpenAI WebRTC connection actually works

The direct WebRTC flow skips the signalling server you would normally build. There is no WebSocket handshake to negotiate the call; OpenAI uses plain HTTP for the SDP exchange (OpenAI WebRTC guide, accessed 2026-07-17).

It runs in four moves, and the code below is the complete flow. We executed it end-to-end against the live API on 2026-07-17; the field names, status codes, and connection states that follow were observed.

First, your backend mints a short-lived client secret so your real API key never touches the browser.

// server.js (Node 18+) — your standard API key stays server-side.
const r = await fetch("https://api.openai.com/v1/realtime/client_secrets", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    session: {
      type: "realtime",
      model: "gpt-realtime-2.1-mini",        // pin a model and date it
      audio: { output: { voice: "marin" } },
    },
  }),
});
const token = await r.json(); // → { value: "ek_…", expires_at, session }
// hand token.value to the browser; it expires quickly by design

Second, the browser creates an RTCPeerConnection, attaches the mic, opens the events channel, and POSTs its raw SDP offer to OpenAI — which returns the SDP answer in the HTTP response body with a 201 Created.

// browser — fetch the ephemeral key from YOUR backend, never OpenAI directly
const { value: EPHEMERAL_KEY } = await (await fetch("/token", { method: "POST" })).json();

const pc = new RTCPeerConnection(); // note: no iceServers passed — this is the whole point

pc.ontrack = (e) => { audioEl.srcObject = e.streams[0]; };        // model audio out
const mic = await navigator.mediaDevices.getUserMedia({ audio: true });
pc.addTrack(mic.getAudioTracks()[0], mic);                        // your mic in

const events = pc.createDataChannel("oai-events");                // JSON events channel
events.onmessage = (e) => {
  const ev = JSON.parse(e.data); // session.created, response.*, input_audio_buffer.*
  if (ev.type === "response.output_audio_transcript.done") console.log(ev.transcript);
};

await pc.setLocalDescription(await pc.createOffer());
const resp = await fetch("https://api.openai.com/v1/realtime/calls", {
  method: "POST",
  body: pc.localDescription.sdp,
  headers: { Authorization: `Bearer ${EPHEMERAL_KEY}`, "Content-Type": "application/sdp" },
});
await pc.setRemoteDescription({ type: "answer", sdp: await resp.text() }); // 201 + answer SDP

Third, session updates, tool calls, and transcripts flow as JSON over the data channel named oai-events, using the same schema as the WebSocket API. Fourth, audio is just a normal media track in each direction (OpenAI WebRTC guide, accessed 2026-07-17).

Here is what our live run observed, in order: the secret minted as { value, expires_at, session }; the SDP exchange returned 201 Created; ICE went checking → connected with no ICE servers configured; oai-events opened; and the model answered our first response.create out loud. OpenAI's server-side voice activity detection then took further turns off the incoming audio stream — the full round trip, working.

Notice what is missing: you never pass iceServers to that RTCPeerConnection. That omission is deliberate, and it is the key to the entire TURN question.

The architecture question: where do STUN and TURN come in?

On the direct browser-to-OpenAI path, STUN and TURN do not come in at all — from your side. OpenAI terminates WebRTC server-side at publicly reachable endpoints and returns host candidates only, with no STUN or TURN server (webrtcHacks teardown of the GA gpt-realtime stack, dated 2025-09-23).

The teardown found OpenAI advertising multiple public Azure datacenter endpoints and connecting clients directly to them over UDP on port 3478 and TCP on port 443 — with 443/TCP added at GA specifically to pass firewalls that block UDP and non-web ports (webrtcHacks, 2025-09-23).

That design has a clean consequence. Because OpenAI's endpoint is public and ships its own TCP/443 fallback, the browser-to-OpenAI hop traverses most NATs and many corporate firewalls without any TURN server on your side. Our executed run is the proof in miniature: ICE reached connected with no ICE servers configured at all.

And you could not add one if you wanted to. OpenAI controls the answer SDP, so there is no place to inject your relay. If WebRTC muscle memory has you reaching for an iceServers block here, there is nothing for it to do — on this topology a TURN server is just not necessary

This is why direct-path failures reported in OpenAI's community forums read as transient service issues, not NAT problems — the direct path rarely fails on NAT because OpenAI engineered the firewall escape hatch into its own endpoint.

When you DO need TURN: the moment you own a WebRTC leg

Here is the turn. Most production voice agents do not send browser audio straight to OpenAI. They insert a server in the middle — and that server changes everything about connectivity.

Why add a server? To keep your API key off the client, add tools and guardrails, run server-side voice activity detection and barge-in, record or transcribe, bridge telephony, or swap the model provider without shipping a new client. All sensible reasons — and all of them create a second WebRTC connection that you own.

On the browser-to-your-server leg, you are the WebRTC endpoint. That means you own NAT traversal. A public-IP media server handles most users through host and server-reflexive candidates — but users on symmetric NAT, UDP-blocked corporate, hospital, or bank networks, or restrictive Wi-Fi cannot connect without a TURN relay, ideally TURN over TLS on port 443 so it looks like ordinary HTTPS.

This is not my claim alone. Python's aiortc uses a standard RTCConfiguration with iceServers, and the same NAT rules apply on its browser-facing leg.

So the decision is not "does OpenAI Realtime need TURN." It is "does my architecture put a WebRTC leg under my control." Here is that decision as a table.

The Python and server-side path

Python builders hit this split constantly, so it deserves its own section. There are two very different Python paths, and only one of them touches ICE.

If your server already has the audio — from a telephony system, a media pipeline, or a worker — use the WebSocket transport with the openai Python SDK. There is no browser peer, no ICE, and no TURN.

If your Python service must be a WebRTC peer — for example a headless agent that browsers connect to directly — you use aiortc, "WebRTC and ORTC implementation for Python using asyncio" (aiortc, accessed 2026-07-17). Now you own the browser-facing leg, and you are back in topology C: STUN and TURN required.

Configuring ICE in aiortc is a standard RTCConfiguration. Point it at your relay, preferring TURNS on 443 for locked-down networks:

from aiortc import RTCConfiguration, RTCIceServer, RTCPeerConnection

config = RTCConfiguration(iceServers=[
    RTCIceServer(urls="stun:openrelay.metered.ca:80"),
    RTCIceServer(
        urls="turns:openrelay.metered.ca:443?transport=tcp",
        username="<from your TURN credential API>",
        credential="<short-lived secret>",
    ),
])
pc = RTCPeerConnection(configuration=config)  # your browser-facing peer now has a relay

This is exactly Metered Python SDK fits. metered-realtime (PyPI v1.0.0, async, built on aiortc) is the SDK for building that browser-facing WebRTC leg in Python, and it auto-injects Open Relay TURN so a Python agent that peers with browsers gets NAT traversal without you standing up coturn.

To be precise about what it is: metered-realtime is the transport layer under your agent, not an OpenAI Realtime client. Your agent still talks to OpenAI over WebSocket or WebRTC; metered-realtime handles the browser-facing WebRTC peer and its relay.

Why your voice agent fails on office and hospital Wi-Fi

This is the failure that many devs face, and it maps exactly onto the topology table. When a WebRTC AI agent "works on my machine" but dies on a customer's corporate network, the broken leg is almost always the one you own.

Corporate, hospital, and bank networks block outbound UDP and non-standard ports, and many run deep packet inspection that drops traffic on 443 that is not genuine TLS. Symmetric NAT breaks the direct peer path on top of that. Your host and server-reflexive candidates all fail, and the call never connects.

TURN over TLS on port 443 is the escape hatch. It performs a real TLS handshake and looks identical to an HTTPS request, so it survives both the firewall and the DPI. For a deeper treatment of why 443 and TURNS specifically are what get through. you can test the TURN over TLS in TURN server testing tools like: TURN server testing

Latency matters here too, because this is voice. A relay three regions away adds audible delay, so a production TURN service with relays near your users — not a single box — is what keeps relayed calls sounding real. This is the same relay reality behind every TURN server for AI agents

agent audio is real-time media, and real-time media behind a corporate firewall needs a good relay. And when a relayed call still drops mid-session — networks change, Wi-Fi roams — WebRTC reconnection handling is what gets the user back without a page refresh.

The fix has two speeds. Open Relay gives you 20 GB/month of free TURN with ports 80, 443, and TURNS out of the box

METERED TURN — for the leg you own (facts dated 2026-07-17)

• Metered TURN product: 500 MB free trial, then paid.

Production tiers (https://metered.ca/stun-turn, verified 2026-07-03)

• Growth $99 / 150 GB, Business $199 / 500 GB, Enterprise $499 / 2 TB, custom above. Usage is metered as ingress + egress.

• 31+ regions, 100+ edge PoPs for low-latency relayed voice.

• Ports 80 / 443 / TURNS, dynamic per-session credentials, per-credential analytics, 24/7 human support.

Metered's managed TURN service is the same relay function across 31+ regions with fixed, allowlistable IPs and region pinning — the connectivity most enterprise voice deployments end up needing.

What OpenAI Realtime costs

Cost is the other thing that surprises builders, so here are the current list prices. These are OpenAI's published figures per 1M tokens unless noted (developers.openai.com pricing, accessed 2026-07-17).

For per-minute intuition, user audio runs roughly 600 tokens per minute and assistant audio roughly 1,200 tokens per minute. Independent measurements suggest a typical agent costs around $0.18–$0.46 per minute uncached, dropping to roughly $0.04–$0.10 per minute with prompt caching, trimmed tool outputs, and server-side VAD (third-party 2026 measured-session write-ups, accessed 2026-07-17).

Treat those per-minute figures as independent estimates, not OpenAI's own numbers — methodology varies. The list prices above are the facts; the per-minute ranges are directional.

Putting it together: a reference architecture

Stack the pieces and the production shape is clear. A browser captures audio and connects over a WebRTC leg to your backend; your backend runs VAD, tools, and guardrails, then talks to OpenAI; and a TURN relay sits on the browser-facing leg for the users who need it.

You own two things in that picture that OpenAI does not give you: the browser-facing WebRTC leg (which needs TURN) and the signalling for it. If you would rather not wire the backend leg yourself, our free, open-source SDK LLMRTC (@llmrtc/llmrtc-core, -backend, -web-client; Apache 2.0) is a batteries-included version of this backend — browser ⇄ WebRTC ⇄ Node backend ⇄ providers.

LLMRTC is provider-agnostic and lists OpenAI among its supported providers — its OpenAILLMProvider, OpenAIWhisperProvider, and OpenAITTSProvider are swappable by config (llmrtc.org, accessed 2026-07-17) — and its own docs recommend Open Relay TURN for production. Built by our team, it is the "don't hand-roll the media backend" option for a build AI voice agent project.

One more piece you own: signalling for that browser-to-server leg. If you build it yourself, Metered Realtime is free managed signalling with an MIT-licensed open-source client, so you can start free instead of standing up your own WebSocket layer. It is the natural companion to the relay — the two things OpenAI's direct path handles for you, and you handle yourself the moment you own a leg.

That is the whole architecture in one honest sentence: OpenAI gives you the model and a public endpoint; you give yourself the media leg, its relay, and its signalling — and TURN lives on that leg, not on OpenAI's.

Frequently asked questions

Does the OpenAI Realtime API need a TURN server?

Not on the direct browser-to-OpenAI path. OpenAI terminates WebRTC at public endpoints with host candidates only and a TCP/443 fallback, so that leg traverses most networks without your TURN — and you cannot add one (webrtcHacks, 2025-09-23). You need TURN the moment you own a WebRTC leg, such as browser to your media server, where users on restrictive networks fail without a relay.

WebRTC or WebSocket for the OpenAI Realtime API?

Use WebRTC for browser and mobile clients that capture or play audio directly, and WebSocket when your server already has raw audio from a media pipeline, call system, or worker (OpenAI guidance, accessed 2026-07-17). WebRTC is the client-edge transport and involves ICE; WebSocket is the server-side transport with no ICE and no TURN.

Why does my OpenAI Realtime WebRTC agent fail on a corporate network?

Because the failing leg is one you own, not the OpenAI leg. Corporate, hospital, and bank networks block UDP and non-standard ports and inspect port 443, so your browser-to-your-server WebRTC connection cannot use host or server-reflexive candidates. TURN over TLS on port 443 is the fix — it looks like ordinary HTTPS and survives deep packet inspection.

How do I connect to the OpenAI Realtime API from Python?

Two ways. If your server already has the audio, use the openai Python SDK over WebSocket — no ICE, no TURN. If your Python service must be a WebRTC peer that browsers connect to, use aiortc with an RTCConfiguration that includes STUN and TURN ICE servers, because you now own NAT traversal on the browser-facing leg.

Is the OpenAI Realtime API generally available, and which model should I use?

Yes. It reached GA on 2025-08-28 with gpt-realtime (OpenAI, accessed 2026-07-17). As of 2026-07-17 the catalog lists gpt-realtime-2 as default with gpt-realtime-2.1 and gpt-realtime-2.1-mini as current point releases. Pin a specific model and date it, because OpenAI ships new realtime models every few months.

How much does the OpenAI Realtime API cost per minute?

OpenAI prices gpt-realtime-2.1 at \(32 per 1M audio-input tokens and \)64 per 1M audio-output tokens, with the mini at $10 and $20 (developers.openai.com, accessed 2026-07-17). Independent 2026 measurements suggest roughly $0.18–$0.46 per minute uncached, falling to about $0.04–$0.10 with caching and trimmed outputs — estimates, not OpenAI figures.

The bottom line

The OpenAI Realtime API over WebRTC does not need a TURN server on the direct path — OpenAI built the firewall escape hatch into its own public endpoints, and you cannot add your own relay there. That is the part existing guides simply do not cover.

But production voice agents put a server in the loop, and that creates a WebRTC leg you own. On that leg, users behind symmetric NAT and UDP-blocked enterprise networks fail without TURN over TLS on port 443 — the same connectivity problem every serious WebRTC app eventually meets.

So build the direct path when you can, and the moment you own a media leg, put a real relay under it: free on Open Relay, or move to Metered's managed TURN service when you need 31+ regions, fixed IPs, and per-session credentials for relayed voice that actually connects.

About the author: This guide was written by James Bordane an Open Source enthusiast

Does an AI voice agent need a TURN server? Usually yes — and the precise reason matters more than the slogan.

A voice agent connects a user's browser to a model in the cloud over WebRTC. Many users connect directly, but anyone on a corporate network that blocks UDP can only reach the agent through a relay on TCP port 443.

That block is per-network, not per-session — so for those users, a TURN server for AI agents isn't optional. Their media is 100% relay-dependent.

That's the honest version of a claim you've probably seen stated as a flat absolute. Let's fix the absolute, then show you exactly when — and how — to wire the relay in.

TL;DR: AI voice agent media travels browser-to-cloud over WebRTC, not peer-to-peer, so when the direct path is blocked a relay is the only fallback. Most home users connect directly, but corporate firewalls that block UDP force every session onto a TURN relay over TCP/443, and some platforms (AWS Bedrock AgentCore) mandate TURN outright.

How an AI voice agent actually connects

Start with the shape of the connection, because everything downstream follows from it.

A voice agent is not a peer-to-peer call. It's a browser talking to a model running on a server in the cloud, over a single WebRTC connection. Your microphone audio flows up; the agent's synthesized voice streams back down.

That means there is exactly one remote endpoint — the cloud — and exactly one non-direct option if the direct path fails: a relay.

Compare that to a human-to-human call, where two peers can sometimes find a local network path to each other, or fall back through a relay if not. An agent has no second peer to try. There's no LAN-local shortcut, no alternate route — just the cloud endpoint and whatever path that can reach it.

So the relay isn't a nice-to-have you bolt on for edge cases. It's the only insurance the architecture leaves you when the direct path is blocked.

How much agent traffic actually needs a relay?

It depends on the network — and that dependence is the whole answer. Relay usage varies widely: some users never touch a relay, others can't connect without one.

The cloud agent has a public IP, so a user on an open home connection usually reaches it directly — no relay involved at all. Some vendor marketing rounds this up to an absolute, but that skips the users who connect straight through.

The three cases where a relay is the only path

Here's the claim worth carrying. Not "all agent traffic is 100% relay" — instead, three specific, defensible cases where a voice agent's media is fully relay-dependent, and a fourth reality that ties them together.

One: no second peer, no shortcut. Because the agent has a single cloud endpoint, a relay is the only non-direct path. When the direct route fails, there is nothing else to try

Two: UDP-blocking is binary per network. Most home users on open UDP connect straight to the agent — no TURN needed. But a locked-down corporate or enterprise LAN blocks UDP entirely and allows outbound traffic on only a few ports. There, TURN over TLS on port 443 "is often the only path that gets through, because that port looks like ordinary HTTPS traffic". For that population it isn't 15% — it's effectively 100%, because the block is per-network, not per-session.

Three: some platforms mandate TURN by architecture. AWS Bedrock AgentCore's WebRTC runtime states flatly that "TURN relay is required for media traffic between the client and the agent," and offers Amazon KVS as managed TURN (AWS docs, 2026-03). On those platforms, TURN is part of the connection path regardless of the user's network.

And the fourth reality: because you can't predict which of your users sits behind a UDP-blocking firewall or a carrier-grade NAT, you provision the relay for all of them. The agent that "works in the demo" is the one that skipped this step and hasn't met a corporate user yet.

The pattern we see: works on your network, breaks on mobile and in the office

A developer's voice agent works perfectly on their local network. Then they connect a device over a mobile phone, or from inside an office network — and it fails. Audio never arrives, even though the app says "connected."

Nearly every time, the cause is the same: TURN wasn't configured properly. Once we walk the team through a correct TURN setup, the agent works everywhere — on mobile networks as well as inside offices.

Two mechanisms sit behind that one symptom. Mobile networks typically run carrier-grade NAT (CGNAT), which behaves like symmetric NAT and makes the address STUN discovers unusable to the far side. Office and campus Wi-Fi blocks UDP outright at the firewall.

Different mechanism, same result: the direct path dies, and only a relay on TCP/443 survives.

The reason this is so easy to miss is that your development machine is the one environment where none of it applies. On localhost there's no network to cross. Ship to real users on real networks, and the relay is suddenly load-bearing.

TURN requirement by deployment scenario (2026)

Rather than argue percentages, here's the decision laid out by scenario. Each row is dated and sourced, so you can map your own deployment to a row and know where you stand.

TURN requirement by deployment scenario — verified 2026-07-15:

Read down the "TURN relay needed?" column and the pattern is obvious. The only rows where you can skip a relay are the open-home case and the hosted platforms that quietly run their own relay for you. Every self-hosted or enterprise-facing deployment lands on "yes."

How to wire TURN into your voice agent

The mechanism, then the config. WebRTC gathers connection candidates, and you hand it a list of ICE servers to try: STUN discovers your public address, and TURN relays your media when a direct path is impossible. If those three acronyms are fuzzy, this STUN vs TURN vs ICE explainer is a clean primer.

For a voice agent, you pass an iceServers array into your peer connection — the same shape whether you use raw WebRTC or a framework. A minimal config includes a STUN entry and a TURN entry with credentials:

const iceServers = [
  { urls: "stun:<your-stun-url>" },
  {
    urls: [
      "turn:<your-turn-url>:443?transport=tcp",  // TCP/443 survives UDP-blocking firewalls
      "turns:<your-turn-url>:443",               // TURN over TLS, looks like HTTPS
    ],
    username: "<short-lived-username>",
    credential: "<short-lived-credential>",
  },
];

const pc = new RTCPeerConnection({ iceServers });

Two details do the heavy lifting. The transport=tcp on port 443 is the entry that gets through corporate firewalls, and the turns: (TURN over TLS) entry makes that traffic look like ordinary HTTPS. Serve both, and the strict-network users from the scenario table can finally connect.

For debugging, one setting is worth knowing: iceTransportPolicy: "relay" forces every candidate through TURN. Set it during testing to confirm your relay path works before real firewalls are in the picture — if it connects with relay forced, your locked-down users will connect too.

There's a second piece your agent needs, and it's easy to forget in the media excitement: a signalling channel to exchange those SDP offers and ICE candidates in the first place. That's the "and how do the two sides find each other?" question. Metered Realtime provides managed signalling free — 100 concurrent connections and 100,000 messages a month, with an MIT-licensed open-source client — so the same vendor covering your relay can cover the control channel too, without a second integration.

If you'd rather see this end to end, we built and measured a full TypeScript voice agent — mic to model and back, with the TURN step wired in — in this build-a-voice-agent walkthrough. It's the build-side companion to this infrastructure piece.

TURN for self-hosted LiveKit Agents

If you're running LiveKit Agents self-hosted, the NAT-and-firewall reality is exactly the same — your media crosses the same hostile networks as any other WebRTC app, and remote users behind UDP-blocking firewalls need a relay to reach your deployment.

LiveKit's server can run an embedded TURN service, but many teams point a self-hosted deployment at an external, multi-region TURN service for production reach and redundancy. You do this by supplying external TURN URLs and short-lived credentials in the server's ICE/TURN configuration, so every client LiveKit provisions receives relay candidates on TCP/443 alongside the usual STUN and UDP options.

The wiring is the same principle as the iceServers block above — a STUN entry plus a turn:/turns: entry on port 443 — just applied at the LiveKit-server layer instead of per peer connection. Point it at a relay with broad regional coverage and 24/7 support, and your self-hosted LiveKit Agents deployment inherits the enterprise-firewall traversal it needs.

Both Metered's TURN service and the free Open Relay network slot in here as that external TURN target.

TURN is table stakes — the honest bottom line

Here's the position we'll stake our name on, after a decade of running relays for other people's WebRTC.

TURN is table stakes when you're working with WebRTC. Many peer-to-peer connections simply don't hold up in the real world — especially when it matters most — because of NAT and firewall rules.

Mobile networks sit behind CGNAT. The Wi-Fi inside hospitals, schools, and offices blocks the direct path. Those are precisely the places your agent will be used.

So the pragmatic move isn't to debug the failing 15% after launch. It's to provision the relay from day one, offer it on TCP/443 with TLS, and mint short-lived credentials — then stop thinking about NAT and ship. The relay is cheap insurance against the exact users you most want to impress.

"100% relay" was never the right way to say it. "You cannot ship a production voice agent without a relay for the users whose direct path is blocked" — that's the truth, and it's enough.

Frequently Asked Questions

How much AI voice-agent traffic actually needs a TURN relay?

It varies widely by network, so no single percentage fits. Users on open home connections often reach the agent's public-IP server directly, with no relay at all.

On UDP-blocking corporate networks, every session needs a TURN relay over TCP/443, because that's the only path out. Some platforms also mandate TURN regardless of the user's network.

Does the OpenAI Realtime API need a TURN server?

OpenAI's hosted Realtime service handles connectivity itself — it engineered a private relay-transceiver over ICE-TCP and skips classic TURN (InfoQ, 2026-05-20). But if you self-host the agent instead of using OpenAI's endpoint, you don't have their global relay network. You add a TURN server on 443 to get the same firewall traversal for your own users.

Why does my voice agent work locally but fail for real users?

On localhost there's no network to cross, so WebRTC connects trivially. Real users sit behind NATs and corporate firewalls that block direct UDP — mobile CGNAT and office Wi-Fi are the usual culprits. Without a TURN relay carrying media over TCP/443, the connection has nowhere to go, and audio never reaches the agent despite a "connected" status.

Why does WebRTC fail on corporate networks?

Corporate firewalls commonly block UDP and allow outbound traffic on only a few ports. WebRTC's default UDP media path can't get out, so the connection fails silently. The fix is a TURN server offering TURN over TLS on port 443, which looks like ordinary HTTPS traffic and passes straight through the firewall

Do managed agent platforms require TURN?

Some do, by architecture. AWS Bedrock AgentCore's WebRTC runtime states that TURN relay is required for media between the client and the agent, and offers Amazon KVS as managed TURN (AWS docs, 2026-03). On those platforms TURN isn't optional — it's part of the connection path regardless of the user's network.

coturn or managed TURN for a voice agent?

coturn is free software, but you own TLS certificates, credential rotation, bandwidth, DDoS exposure, and patching across regions. Managed TURN handles all of it with global coverage and an SLA. Self-host if you have the ops capacity and volume; use a managed relay to ship reliably without running relay infrastructure yourself.

Getting started with Metered TURN for voice agents

If you'd rather not run relays across a dozen regions, Metered's TURN service is built for exactly the deployment reality above.

You get 31+ regions and 100+ edge PoPs, TURN on ports 80 and 443 with TURNS/TLS, dynamic short-lived credentials, and 24/7 human support from a team that has operated production TURN, STUN, and signalling for a decade.

About the author: This guide was written by James Bordane, a developer, and open source enthusiast and network engineer