Open-Source Voice Isolation Models Just Got Practical

    On November 18, 2026, Inworld put a 64-million-parameter voice isolation model into the open. And the small parameter count is the whole story. Voice Isolator is the company’s first open-source release.

    Code and weights, Apache 2.0. Landing in a category that barely registers in the rankings.

    Run open-source voice isolation for local AI today and two real choices exist. Inworld Voice Isolator works at 48 kHz in two checkpoints: voice-isolator-v1 for offline cleanup at 64.0M parameters. And voice-isolator-v1-realtime for live streaming at 58.2M parameters with 63 ms of algorithmic latency at 20 ms look-ahead. VoxLock does target-speaker extraction for phone calls, picking the caller out of the mix on its own, streaming on a CPU at 24 ms of latency, with hooks into Pipecat and LiveKit.

    A third signal.

    A paper training causal masking transformers at 140M, 243M, and 374M parameters with 20 ms of latency.

    Marks where the whole category is heading.

    Why Voice Isolation Runs Before Your Transcriber

    Went looking for rankings of open voice models recently.

    Found speech-to-text leaderboards, text-to-speech comparisons, and isolation as a footnote if it appeared at all. That ordering is backwards. Because the step between raw audio and your transcriber is where accuracy dies.

    And it is the one step nobody prices.

    Name the operator problem and it sounds mundane.

    Transcript quality in your agent inherits from audio you never controlled: someone calls from a loud room, a radio runs in the kitchen, a second voice talks across the first.

    Transcripts come back garbage, and the reflex move is swapping STT providers or paying for a bigger model. Cleaning the audio before the transcriber hears it costs less. And that is the entire job of these two models.

    When did you last actually listen to your own worst call recording?

    Me neither.

    Local deployment turns that fix from optional into mandatory. Keep call audio on your own hardware. Privacy, cost, whatever the reason. And the isolation stage has to run there too, on machines you already own. Parameter counts matter more here than in text for exactly that reason: a 58.2M checkpoint and a 27M-parameter fine-tune slot into hardware where a multibillion-parameter model does not.

    Inworld Voice Isolation in Two Checkpoints

    Voice Isolator keeps the voice roughly as recorded, timbre and breaths and level intact, while stripping background chatter, music, and transients. That preservation detail matters more than it sounds. Noise removal done aggressively flattens a voice until a transcriber handles it worse, not better. And plenty of tools are built to sound clean to a human ear rather than to keep what speech-to-text needs.

    This one was built for the transcriber.

    The offline model, voice-isolator-v1, sees both directions: every frame gets its whole 8-second chunk. And longer inputs render in 8-second chunks with 2 seconds of overlap. The realtime model, voice-isolator-v1-realtime. Whoever named it never typed it into a config file twice. Is causal, trained with a look-ahead of 0, 20, or 40 ms. And its algorithmic latency lands at 63 ms at 20 ms of look-ahead. Sixty-three milliseconds.

    That is a number you can actually put in a pipeline budget, rarer than it should be in this space.

    Installation is refreshingly dull: `pip install inworld-voice-isolation`.

    Code, an interactive demo, and the benchmark all sit in the inworld-ai/voice-isolator repo. Apache 2.0 covering code and weights means it goes inside a shipped product without a single licensing email, which is the only license gate I pay attention to.

    Two honest caveats.

    Both checkpoints work at 48 kHz. So test against your real source audio before trusting them on whatever your pipeline actually ingests. I have not benchmarked them on my own call recordings yet either. So instead of inventing a score, I am sending you to the benchmark in the repo. My first guess was that the realtime checkpoint would be the weak sibling; the gap between 58.2M and 64.0M is small enough that I stopped assuming.

    And I still have not measured, so treat that as a guess, not a verdict.

    VoxLock: Caller Extraction On A CPU

    VoxLock handles the harder half of the problem. The repo describes an open-source voice isolation model that, from one input. The call audio itself. Keeps the caller’s voice and drops every other voice in the room before speech-to-text hears any of it. No enrollment step. No clean reference sample of the speaker handed over up front. The model works out who the caller is from the call itself.

    Per the paper behind it, the author fine-tuned REAL-TSE. A public causal target-speaker extraction model at 27M parameters. On one GPU. One GPU. Hobbyist-scale budget, for a component that used to be something you bought from a vendor. The release bundles ONNX graphs, a Python package.

    And per-file evaluation scores. And streaming runs on a CPU at 24 ms of algorithmic latency, or in the browser.

    The README’s workflow is a single command: `python isolate.py –ckpt ckpt –in call.wav –out caller.wav`, with weights pulled via `huggingface-cli download arbazsiddiqui/voxlock model.safetensors config.yaml`. Pipecat and LiveKit both have integration points. So an agent built on either framework gets this dropped into the stack it already runs. That gap. Between a paper you admire and a thing you wire in on a Friday afternoon.

    Is the whole difference.

    The two models do not compete. Inworld takes out sound that is not a voice; VoxLock takes out voices that are not the caller. Choose by failure mode, not by benchmark score.

    | Model | Size | Latency | Runs on | License | Best for |
    |—|—|—|—|—|—|
    | Inworld voice-isolator-v1 (offline) | 64.0M params | batch; 8 s chunks, 2 s overlap | downloadable weights | Apache 2.0 | cleaning existing recordings |
    | Inworld voice-isolator-v1-realtime | 58.2M params | 63 ms at 20 ms look-ahead | streaming, downloadable weights | Apache 2.0 | live voice pipelines |
    | VoxLock | fine-tune of the 27M REAL-TSE | 24 ms | CPU and browser | open source | phone calls, caller extraction |

    The third signal is the trend line. That work trains causal voice isolators as masking transformers at 140M, 243M. And 374M parameters, all holding 20 ms of latency, with the caller enrolled from the call itself. Independent efforts are converging on the same shape: small, causal, no enrollment, latency in the tens of milliseconds. A category agreeing on its shape this fast means a boring preprocessing step is becoming its own product class.

    Skip the leaderboard-chasing.

    The move is to stop feeding your current STT garbage. Both models cost nothing to try, both ship as downloadable weights. And local execution keeps call audio off other people’s servers. Privacy and cost arguments finished in one sentence. Take the worst recording you own, push it through VoxLock’s isolate.py, and listen. Same file through the offline Inworld checkpoint next, then compare what your transcriber makes of each version. Better transcripts on your worst audio means you just found the cheapest accuracy gain in voice AI this year. Do it this week, before another dollar goes to a bigger model. Your transcriber does not want more parameters; it wants cleaner input.

    Voice Isolation FAQ

    What is the difference between voice isolation and target-speaker extraction?

    Voice isolation removes everything that is not a voice. Music, transients, background chatter. And keeps every human in the recording. Target-speaker extraction goes further: it keeps one voice, the caller, and discards the other humans along with the noise.

    Inworld Voice Isolator covers the first job, VoxLock the second.

    Can these run on CPU?

    VoxLock, yes. The repo has it streaming on a plain CPU at 24 ms of algorithmic latency. And in the browser too. The Inworld checkpoints ship as downloadable weights; the README does not state a minimum GPU. And I have not run them yet, so I will not invent a number here.

    Sources

    – Introducing Voice Isolator — Inworld blog: https://inworld.ai/blog/introducing-voice-isolator
    – Inworld Voice Isolator README — Hugging Face: https://huggingface.co/inworld/voice-isolator/blob/main/README.md
    – inworld-ai/voice-isolator — GitHub: https://github.com/inworld-ai/voice-isolator
    – VoxLock — GitHub: https://github.com/arbazsiddiqui/voxlock
    – VoxLock paper: https://arxiv.org/html/2610.40727v1
    – Causal masking transformer paper: https://arxiv.org/html/2610.35272

    Leave a Reply

    Your email address will not be published. Required fields are marked *