The first time you plug a speech-to-speech model into a mic, it feels like a magic trick. You talk, it talks back, no lag, no robotic cadence, it even laughs at the right moment. Everyone leans in. Someone says "ship it." OpenAI's Realtime API, Google's Gemini Live, Alibaba's Qwen-Omni-Realtime, built for exactly this moment, and they earn it. One model, one WebSocket, audio in, audio out. No ASR-to-LLM-to-TTS relay race, no glue code.
Then it goes to production, and a few weeks in, two things show up that nobody saw in the demo.
The first is the bill. S2S models don't have a caching layer the way text-based LLM calls do: there's no cheap re-send of a system prompt or history, because the model consumes a continuous stream of audio tokens for as long as the session is open, not just when someone's talking. Every extra turn drags the whole conversation's audio context along with it, so a five-minute call costs meaningfully more than a one-minute call, in a way that's hard to predict or cap ahead of time. A cascaded pipeline (separate ASR, LLM, TTS) can cache the repeated parts (system prompt, tool schema, earlier turns) the way any text-based LLM app does, so cost per turn flattens as the conversation runs longer instead of climbing with it.
Relative cost as a conversation gets longer
Illustrative, exact per-vendor cost curves aren't publicly disclosed. Scrub the slider.
Turn 1, S2S no caching: index 1.0
The second thing is worse: something breaks, and you can't tell why. In a pipeline, every stage leaves a paper trail: a transcript, a visible LLM response, a TTS log. When a call goes sideways, you read the trail and know in ten seconds whether it was a mishearing, a bad response, or a mangled word. With S2S, audio goes in, audio comes out, and the reasoning in between is invisible. Debugging means listening to a recording and guessing. Fine for a demo call; a real liability on a support line handling anything regulated, or any dispute over what was said.
Add to that: S2S handles interruptions and turn-taking inside its own black box (sometimes gracefully, sometimes leaking stale audio mid-response), tool-calling is less reliable when the reasoning never touches text, and switching vendors later means re-architecting the whole voice layer, no swapping just the TTS component like you could in a pipeline. It's one vendor, all the way down.
None of this makes S2S a bad choice. S2S is still the right call for some products, the point is knowing the cost curve before you commit, not avoiding S2S outright. It makes the choice between S2S and a pipeline a lot more consequential than "which one felt better in the demo," and a lot harder to reverse once you've already built on top of it. The variables that actually decide it (real call volume, compliance exposure, how much tool-calling the agent needs to do mid-conversation, how fast you'll need to swap a vendor later) rarely line up neatly, and getting the call wrong doesn't show up until you're deep into a rebuild, not a patch.
Picking between S2S and a pipeline isn't a matter of taste, it's a matter of matching the architecture to what the product actually has to survive: real users, real call volume, real bills. Dissolve Labs builds with both, and we make that call by modeling your actual use case against both architectures before a line of code gets written, not by defaulting to whichever one is trending in this month's demo reel.