Safety alignment trained on text does not fully transfer to voice. Audio-native attacks, and joint audio-text attacks that exploit the seam between modalities, walk through guardrails that hold firmly against the same content in writing.
A team takes a text model with a serious safety record — red-teamed for months, refusal-tuned, holding firm against the known catalogue of text jailbreaks — and gives it a voice. Maybe they wrap it in a speech-to-text front end and a text-to-speech back end; maybe they move to a speech-to-speech model that takes audio in and returns audio out. Either way the reasoning is the same, and it is never said out loud: the safety transfers, because it is the same model underneath, or the same training lineage, and safety is a property of the model. So the voice product ships without a safety pass of its own.
It is not safe. The same request the text model refuses in writing — spoken aloud, or split between what is said and what is typed — comes back answered. Not because the voice layer broke the model, but because the safety the team was counting on was trained in one modality and the attack arrived in another.
This post is about that gap: what the audio jailbreak literature has measured, why a joint audio-text attack is worse than either channel alone, and what a voice agent has to evaluate before it is allowed to claim a safety story.
Safety alignment is not an abstract property a model has. It is a set of behaviors learned from training data, and for a text-aligned model that training data was text. The refusals, the boundaries, the resistance to known jailbreak phrasings — all of it was fit against written inputs. Whether any of it covers audio is an empirical question, and the empirical answer is: partially, and not enough.
JALMBench (arXiv 2505.17568) is the largest measurement of this — 11,316 text and 245,355 audio samples against audio language models — and its finding is the one to internalize: text-based safety measures partially transfer to audio inputs. Partial is the load-bearing word. It is not zero, which is why a voice agent is not instantly broken. It is not complete, which is why a voice agent is not safe. The gap between partial and complete is the attack surface, and it is large enough that JALMBench also reports existing moderation methods give only limited protection.
Audio is a genuinely different input surface, for reasons that survive any amount of shared model lineage. It carries a paralinguistic channel — tone, pace, accent — with no text equivalent. It is represented as codec tokens or continuous features, not characters. And the architecture matters: a cascade has a text checkpoint in the middle, the transcript, where a text classifier can still run; a speech-to-speech model has no transcript at all, no point in the pipeline where the input is ever text. A model can be flawless on written inputs and have a safety posture on audio that nobody has measured, because nobody ran the measurement.
The first half of the threat is attacks delivered purely in audio, and they are not theoretical.
Audio Jailbreak (arXiv 2505.15406) built AJailBench — 1,495 adversarial audio prompts spanning ten policy-violating categories — and an Audio Perturbation Toolkit that generates subtle variants of an attack. The headline result is that leading audio-language models lack consistent robustness against these inputs even when the perturbations are subtle. “Even when subtle” is the part that should worry an architect: it means the attack does not require an obviously corrupted, noisy, manifestly-hostile audio clip that some upstream filter might flag. It can be small.
SPIRIT (arXiv 2505.13541) sharpens that into an attack channel with no text analogue at all. It shows speech language models are substantially vulnerable to jailbreaks delivered through imperceptible noise injection in the audio — a perturbation a human listener does not hear. There is no equivalent move against a text model; you cannot inject imperceptible noise into a string. This is the clean demonstration that audio is not text-with-a-microphone. It is a wider input space, and the extra room in it is room for an attacker.
Put AJailBench’s breadth next to SPIRIT’s depth and the conclusion is firm: audio-native jailbreaks are real, they are subtle, and the text-shaped guardrails a team inherited were never measured against them.
The second half is worse, and it is the half a single-modality defense cannot see.
A voice agent frequently takes input on more than one channel — audio plus a text system prompt, audio plus typed context, audio plus tool output rendered as text. On Optimizing Multimodal Jailbreaks for Spoken Language Models (arXiv 2603.19127) attacks that structure directly. Its JAMA framework jointly optimizes a jailbreak across both modalities at once — gradient-based perturbations to the audio and the text simultaneously — and the result is decisive: jailbreak success rates 1.5x to 10x higher than unimodal attacks.
The mechanism is the seam. The model fuses the audio and text streams into one internal representation and reasons over the fused thing. But guardrails, when they exist at all, tend to be per-channel: a text classifier on the text, an audio check on the audio. A joint attack puts half its payload in each channel. Each half, inspected alone, is below threshold — neither classifier fires. The hostile input only exists after fusion, which is exactly where no guardrail is looking. A defense that is the sum of two single-modality defenses does not defend the seam between them, and the seam is now a measured 1.5x-to-10x multiplier on attacker success.
Audio-side defenses exist, and the recent ones are good. They are also, like every guardrail, a way of lowering a number rather than retiring it.
ALMGuard (arXiv 2510.26096) takes an elegant route: rather than bolt on an external classifier, it finds universal triggers that activate the safety mechanisms already latent in the model, and drives the success rate of advanced attacks down to 4.6% on average across four models while preserving utility on legitimate tasks. SPIRIT, beyond its attack, offers a defense: post-hoc patching that modifies model activations at inference time, reporting robustness improvements of up to 99% with no retraining.
Read those two numbers honestly. 4.6% average is a large reduction and it is not zero — against an attacker with unlimited attempts it is a door, not a wall. “Up to 99%” is up-to, which means the realized number depends on the attack and the worst case is below the headline. These are strong defenses and they belong in the stack. What they are not is a reason the system is safe. The audio guardrail is a smoke alarm: it lowers the rate and buys warning, and the architecture still has to hold on the calls where it does not fire.
The corrective is not exotic. It is to stop inheriting a text safety story and to build an audio one.
Red-team in audio, not in text. The catalogue of jailbreaks your model resists in writing tells you nothing about the spoken versions. The red-team suite has to deliver attacks as audio, including subtle and imperceptible-noise perturbations, because those are the ones AJailBench and SPIRIT show land.
Test joint audio-text attacks explicitly. A per-channel evaluation will report two reassuring numbers and miss the JAMA seam entirely. The suite needs attacks whose payload is split across modalities, scored after fusion.
Let the architecture decide your guardrail placement. A cascade has a transcript — a real text checkpoint where text guardrails genuinely run — and a team building a cascade should actually use it rather than assume the model handles it. A speech-to-speech model has no such checkpoint, which means audio-domain guardrails are not optional for it; they are the only guardrails it can have.
Run an audio safety eval in CI, with a number. Refusal behavior on audio is a measurable rate. Measure it, gate releases on it, and re-measure on every model swap — voice safety drifts with a model change the same way agentic reliability does, and an audio safety eval is a sibling of the discipline that scores a conversation rather than a transcript.
Decide what a fired guardrail does mid-call. When the audio guardrail catches an attack in the middle of a live conversation, the agent needs a graceful response, not a crash or a dead line — the same graceful-failure design a voice agent needs for an STT or TTS outage applies to a safety stop.
Before a voice agent is allowed to claim a safety story:
A text model’s safety record is a measurement of one modality. A voice agent runs in another one — usually two at once — and the safety in the modality you did not measure is the safety you do not have.
Safety did not become a property of the model the moment you added a microphone. It is still a property of training data and a modality — and your voice agent is running in the modality nobody trained for.
A membership-inference attack that separates your book from a held-out one is usually reading the calendar, not the weights. A successful attack is not a sound proof — and the null it would need to test against cannot be sampled.
17 min →
Unlearning promises a model can forget specific data without a full retrain. What it delivers is output suppression: a small benign relearning set — or simply quantizing the model — brings the 'forgotten' knowledge back.
14 min →
The EU AI Act requires AI output to be marked in a robust, reliable way. A formal impossibility theorem says strong watermarking cannot exist. Five words in the statute carry the collision.
13 min →