I read this article on XDA Developers about someone cloning their own voice from 5 seconds of audio, on a laptop, with no GPU. It’s a good read … and terrifying : the stuff of science fiction.
(After I’d finished the first version, I started rewatching The Expanse which I just adore - and in the second episode, the detective Miller walks around Julie Mao’s apartment, talking into his ‘phone’ which is TTS’ing her voice so he can talk to her apartment and pretend to be her : get a visitor log, listen to her messages.)
So I - ok, Claude - built voice-break. It’s an Android app, side-loaded onto my phone, that records me talking, sends the audio to a small backend running on a Mac, and gets back the same words spoken in a different voice - including a voice cloned from a few seconds of my own recording.
The flow is simple: pick an engine and a voice, hit record, talk, and the app uploads the audio when you stop. The backend transcribes what you said with faster-whisper, then re-generates the speech in the chosen voice with pocket-tts, and sends the new audio back to play automatically, along with the transcript. There’s a little + button next to the voice list that lets you record yourself reading a short prompt and save it as a new voice - you’re now cloned, and it’s freakishly weird to now record new speech and listen to yourself.
I also built a second, parallel pipeline using Moonshine, which bundles transcription and speech synthesis into one package rather than the two separate models above. It’s quicker but rougher around the edges - a much bigger, more cryptic catalog of voices, and a couple of quirks (a hallucinated stray word here, a garbled first syllable there) that took some digging to track down and fix. It sits alongside the original pipeline as an evaluation, not a replacement, and can be ripped out cleanly if it doesn’t earn its keep.
Under the hood it’s a FastAPI backend in Python, and a Kotlin/Jetpack Compose Android app that records raw PCM audio and draws a live waveform while you talk. The backend started out running on my MacBook while the phone was on the same Wi-Fi, and now runs on an always-on Mac Mini reachable over Tailscale, so I can show it off on my phone wherever I am, not just at home.
Whisper/TTS I am finding is excellent and accurate. The round trip time - from stopping recording to playing back the new speech - is around 10-11 seconds, which isn’t bad at all. Mooonshine is almost as good in quality, and much quicker, 2-3 seconds.
It’s not a productionised product, just a personal project to poke at voice cloning technology and see how good - or how alarming - it’s actually gotten. Turns out: pretty good, and yes, very alarming.
Oh, and yes - this entire article was written by Claude in about 2 minutes. I’d grabbed the pocket-tts library manually, got the basics working, then started afresh with Claude Code. I had a working app that freaked out a colleague in 2.5 hours … and then added the app-based voice cloning and the moonshine library in a further 3. I get Claude to document it’s work as a plan, but also to keep a README.md in the repo fully up to date, so it’s actually really easy to generate blog posts.
Yes, I’m keeping up with the debate about AI text - and I have just re-read it and changed a few things to “sound more like me.”
Truly these are interesting times.