Unyap - On-Device Voice Note Transcription
Some people send voice notes. Some people would rather read them. I am firmly in the second camp, and “let me find somewhere quiet to listen to a 90-second WhatsApp ramble” is not a thing I enjoy doing. The obvious fix is transcription, but every easy option ships your audio off to someone else’s server. A private voice note from a friend is exactly the kind of thing I do not want to hand to a cloud API. So I built Unyap: an Android app that transcribes voice notes to text entirely on the phone, using whisper.cpp. No audio and no text ever leaves the device.
 Words stream in as they decode |
 Copy or share the result |
 One-time model download |
How it works
You share a voice note to Unyap from WhatsApp (or any app, or pick a file in-app), the words
stream in as they are transcribed, and then you select, copy, or share the result. That is the
whole product, and it has no account, no upload, no “processing on our servers”. The app requests
a single permission, INTERNET, and uses it exactly once - to download the Whisper model on first
launch. After that it never touches the network again.
flowchart LR Share[Shared voice note] --> Dec[libopus decode] Dec --> PCM[16 kHz mono PCM] PCM --> VAD[Silero VAD] VAD --> C[6-12 s chunks] C --> W[whisper.cpp on CPU] W -- token by token --> UI[Streaming transcript]
The model is Whisper Small, quantized to q8_0 - about 265 MB, downloaded once from Hugging
Face and cached in app storage. It runs on the CPU through a small JNI bridge, with flash_attn
on and the GPU explicitly disabled (on phones the CPU path is both faster to spin up and far less
of a compatibility minefield than trying to get a GPU backend working across a thousand SoCs).
Streaming words as they decode
This is my favourite detail. whisper.cpp’s normal API hands you text one segment at a time - roughly a sentence - which means you stare at a spinner and then a whole line appears at once. I wanted the word-by-word teletype feel instead, where the transcript visibly grows as the model thinks.
There is no official “give me each token” callback, but there is a logits_filter_callback - a
hook meant for biasing the logits during sampling. It fires on every decoding step with the tokens
decoded so far, so I repurpose it: skip the special ones (anything at or above the end-of-text
token), turn the rest into a string, and push it across JNI into Kotlin as provisional text. When
whisper finalises a chunk, new_segment_callback replaces the provisional text with the real thing.
The first version of this sent one token at a time and appended, which works right up until it
doesn’t. When whisper is unhappy with a result it falls back to sampling at a higher temperature
with five decoders at once, and two things go wrong: the callback fires for every decoder, so the
UI gets five interleaved transcripts, and some of those calls come from worker threads that are not
attached to the JVM, where touching JNIEnv is undefined behaviour. So now only the calling thread
streams, and a token count that goes backwards means “fallback started, clear the partial and wait
for the final segment”:
if (std::this_thread::get_id() != cb->tid) return; // worker thread, no JNIEnv here
if (n_tokens <= cb->last_n_tokens) { // decoding restarted: fallback
cb->partial_enabled = false;
send_text(cb, cb->onPartial, "");
return;
}The text goes over as raw bytes rather than through NewStringUTF, because a token can end halfway
through a multi-byte UTF-8 character, and NewStringUTF does not forgive that.
On the Kotlin side the transcription runs on a background thread and the tokens come out as a
Flow<StreamEvent> built with callbackFlow, buffered so the decoder never blocks on the UI.
Compose just collects the flow and appends. The decode finishes with a Done event carrying the
full text, the elapsed time, and the auto-detected language - language detection is left on
(language = nullptr), so German and English voice notes both just work, along with the 90-odd
other languages Whisper knows.
Decoding the audio first
Whisper wants 16 kHz mono float PCM, and a WhatsApp voice note is none of those things - it is
usually Opus in an OGG container. The first version let Android do it: MediaExtractor +
MediaCodec decode whatever audio/* you throw at it, and I downmixed and resampled to 16 kHz on
the way out. That turned out to be one of the slowest parts of the whole app (more on that below),
so Opus now goes through libopus directly and MediaCodec is only the fallback for everything else.
The one trap worth mentioning is memory. A long recording is a lot of samples, and the obvious
MutableList<Short> boxes every single one into a heap object - roughly 16 bytes to store 2 bytes
of audio, which OOM-crashes the app on anything lengthy. The fix is a
plain growable ShortArray that doubles when full, which keeps it at 2 bytes per sample.
// A MutableList<Short> boxes every sample (~16 bytes each) and
// OOM-crashes on long recordings; a raw ShortArray stays at 2 bytes.
var pcm = ShortArray(1 shl 18) // ~256K samples to startNot crashing on older phones
This is the part I expected to be trivial and absolutely was not. whisper.cpp is fast on ARM
because it leans on modern instruction-set extensions - dotprod, fp16 arithmetic, i8mm. If you
build with a high -march to get those, the resulting binary contains instructions that simply do
not exist on older CPUs, and the app dies with a SIGILL (illegal instruction) the moment it hits
one. Build at the safe baseline instead and you leave a big chunk of performance on the table for
everyone on a recent phone.
The clean way out is to not choose at build time at all. I compile the JNI core at the plain
arm64-v8a baseline (ARMv8.0-A), and let ggml build its CPU variants as separate, dlopen-able
backend libraries:
set(GGML_BACKEND_DL ON CACHE BOOL "" FORCE) # each variant a loadable .so
set(GGML_CPU_ALL_VARIANTS ON CACHE BOOL "" FORCE) # armv8.0 .. armv9.2
set(GGML_NATIVE OFF CACHE BOOL "" FORCE) # never -march=nativeAt startup ggml scores the variants against what the CPU advertises through getauxval(HWCAP)
and loads the best one the device supports. A Pixel 8 gets the i8mm kernels; a five-year-old
phone quietly falls back to the baseline and still runs. Same APK, no SIGILL. (Mostly true. It
turned out “best” and “fastest” are not the same thing - see below.)
A second Play Store hoop: apps targeting SDK 35+ must support 16 KB memory pages, so the native
libraries need their ELF segments aligned to 16 KB. That is a couple of linker flags set before any
target is defined, so every library - mine and all the ggml variants - inherits the alignment.
With -O3, LTO, dead-code stripping and hidden visibility on top, the native side stays lean.
Making it eight times faster
Version 1.0 ran at roughly real time on a Pixel 8: fifteen seconds of audio in about fifteen seconds. Fine for a short note, a bit of a wait for a long one, and the text arrived in big lumps with long silences in between. Version 1.1 transcribes a minute of audio in about six seconds and shows the first words after about one. No new model, no GPU - just finding out where the time actually went.
Measure first
I added a benchmark build type: release-equivalent, installs next to the real
app, and has one extra activity that transcribes a file with a given model and settings and writes
timings and the transcript to a JSON file. A Python script drives it over adb. For accuracy I
stitched sentences from Google’s FLEURS dataset
into fake voice notes from 18 seconds to two minutes, German and English, with known reference
text, so every change gets a word error rate and not just a vibe. Two traps on the way: files adb
pushes into the app’s own external folder are invisible to the app (it now fetches inputs from my
laptop through adb reverse), and a benchmark running behind the lock screen is silently moved to
the background CPU group, which made decoding twenty times slower. The phone has to be awake, and
the activity shows over the lock screen.
Whisper always thinks in 30 seconds
Every call encodes a full 30-second window, padded with silence if the audio is shorter, and the encoder is most of the cost. Worse, with automatic language detection whisper.cpp encodes the first window once to guess the language and then again to transcribe it. The app now cuts the audio into 6-12 second chunks at the quietest point it can find, shrinks the encoder’s context to each chunk (plus four seconds of padding - two was measurably less accurate), and a small patch, applied to a copy of whisper.cpp at build time, reuses the language-detection pass instead of throwing it away. Shorter chunks also fixed the “big lumps” problem: text now arrives every couple of seconds instead of every fifteen.
Little cores make everything slower
The app used every core it had. ggml splits each operation evenly across threads and then waits for the slowest one, so the Pixel’s four small Cortex-A510s held the five fast cores back. Five threads beat nine by 1.4x; six (one little core) was the worst of all. The rule now leaves the little cores alone, unless a phone has so few big ones that using everything wins anyway.
The best kernels were the slowest
I did not expect this one. ggml rates its SVE2 variant
(armv9.0) above the plain NEON + i8mm one (armv8.6), and the Pixel 8 supports SVE2, so that
is what it loaded. Forcing armv8.6 cut the encoder time to a third, with word-for-word identical
output. Phone cores implement SVE with 128-bit vectors, the same width as NEON, so there is nothing
to gain and the SVE code paths are just slower. The app now tries the NEON variants itself, best
first, and the SVE builds are no longer even in the APK.
Decoding audio took longer than transcribing it
With the transcription down to a few seconds,
the “Decoding audio…” screen was suddenly the slowest step. Demuxing a minute-long voice note’s
3,300 Opus packets takes 128 ms. Decoding them through MediaCodec took 2.5 to 7 seconds, because
the codec runs in a separate media process and every 20 ms packet is a round trip. libopus decodes
the same note in-process in 80 ms, straight to 16 kHz, so the resampling step is gone too.
Skipping silence dropped speech
Voice activity detection (Silero VAD, under 1 MB, bundled) removes the pauses before whisper sees them, which saves time and gives the chunker good places to cut. On the English test note it also deleted two whole sentences. Those speakers were recorded at about -62 dBFS, too quiet for the VAD but not for whisper, so their speech never reached the model at all. The VAD now listens to a level-normalised copy of the audio while whisper still gets the original, and the English error rate went from 27% to 7%.
What did not make the cut
The benchmark said no to a 4-bit model (6% faster, more
mistakes), 5-bit models (no fast i8mm path, three times slower), Whisper Base (fast until it got
stuck in a loop repeating itself), Arm’s KleidiAI kernels (faster and wrong), and compiler flags
(-O2 against -O3 made no difference at all).
The result
On a Pixel 8 a minute of voice note now takes about six seconds, the first words appear within a
second, and it still gets there without a single packet leaving the phone. Older phones get the
same pipeline with the instructions they have: forced onto the Pixel, the variant for the oldest
arm64 CPUs produced the same text at half the speed, and an actual old phone adds its slower cores
on top of that. The whole thing is Kotlin + Jetpack Compose over a thin JNI layer, targeting
arm64-v8a, min SDK 29.
If you like the genre of Android app that exists to solve one stubborn problem and nothing else, the other one I built is Brightness Lock, which dims the screen so Chinese metro gates will scan a QR code off a Pixel.
Unyap is on Google Play, and
the signed APK is on the GitHub releases page
if you would rather sideload it. It is open source under MIT, with whisper.cpp and libopus bundled as
submodules, and the benchmark lives in the repo under tools/bench. The code is on
GitHub.