Products#Text to speech#Model inference

Google puts speech recognition and TTS on-device: 100+ languages, 100 ms speech onset

Google brought offline ASR and TTS to AI Edge Gallery: 100+ languages at 10x real time, and a TTS model that starts speaking in about 100 ms at 24 kHz.

Key art from the Google Developers Blog post on on-device speech

Google announced on November 10 that AI Edge Gallery now supports on-device, offline speech recognition (ASR) and text-to-speech (TTS), and can pair both with on-device small models such as Gemini Nano and Gemma 4. Unlike cloud speech APIs, these models run on the device: recognition, synthesis and conversation can happen entirely offline, with audio never leaving the phone.

Key points

  • Recognition: 100+ languages at 10x real time: ML Kit’s GenAI speech recognition reaches 8.0–9.0% word error rate on FLEURS and transcribes about 10x faster than real time (roughly 300 ms for 3 seconds of speech), cutting end-to-end speech-translation wait times by 35–44%.
  • Synthesis: about 100 ms to start speaking: the TTS model is a few-step diffusion model accelerated on the NPU. It begins speaking in about 100 ms, outputs 24 kHz audio at up to 50x real time, offers 8 expressive voices across 21 locales, and the company says a 27-second paragraph synthesizes in about 500 ms using over 50% less power than traditional autoregressive TTS.
  • No APK cost, and a cross-platform path: both models are managed by AICore as part of the OS, adding 0 MB to the app. For cross-platform apps, MediaPipe Tasks and LiteRT provide .litertlm bundles — Parakeet-TDT 0.6B, PocketTTS 100M, Kokoro-82M and Phonon 31M — that run alongside on-device SLMs in a compact memory footprint.

Key art from the Google Developers Blog post on on-device speech

Context

For the past two years, most speech capability arrived as a cloud API: accurate, but network-dependent and billed per call. On-device speech improves three things at once — latency, privacy and cost. Audio stays on the device, recognition does not wait for a round trip, and usage is not metered by tokens.

Google offers two integration paths. ML Kit’s GenAI Speech Recognition and Text-to-Speech APIs are native, Android-only, and managed through AICore, so developers never handle model downloads or updates and the app does not balloon in size. MediaPipe Tasks with LiteRT covers open-weight models on iOS, macOS, Windows, Linux and the web, for products that need cross-platform reach or custom models.

The model bundles form a size ladder worth noting: Parakeet-TDT 0.6B, PocketTTS 100M, Kokoro-82M and Phonon 31M span server-grade accuracy down to lightweight synthesis in tens of megabytes. Teams can pick a combination against a device’s memory and latency budget instead of being locked to one model.

Availability

Android developers can call the ML Kit GenAI speech APIs directly. Cross-platform teams can start from the .litertlm bundles and wire Parakeet, PocketTTS, Kokoro or Phonon into their existing pipeline. To try it firsthand on a phone or desktop, install AI Edge Gallery: its AI Chat now takes microphone input and reads replies aloud, with selectable engines, voices and languages, and highlights sentences as it speaks.

Analysis

  • The numbers are Google’s own measurements: WER, 10x real time and 100 ms onset come from Google’s model and device combinations. Your hardware, language and microphone conditions will differ; test with real audio first.
  • OS-managed models change distribution: models shipping with the OS keep app size flat and updates automatic, but availability then depends on the device and OS version — cross-platform products still need the LiteRT path as a fallback.
  • Speech is only the interface: recognition and synthesis are entry points; a working voice assistant also needs a dialogue model. Google pairs them with Gemini Nano or Gemma 4, which puts it on a different path from self-hosted stacks such as Ming 3.0.

Summary

If your app needs offline transcription, voice input or read-aloud, there are now two paths worth evaluating: ML Kit with AICore on Android, or MediaPipe Tasks with LiteRT open weights for cross-platform. Measure latency and power on your target devices before moving any speech workload off the cloud; Google’s figures describe the ceiling, and the real experience depends on device and language.