It may not be the most well-known use of AI on Android, but it's definitely one of the most useful: Live Caption will do exactly what its name suggests, putting up captions in real time on top of audio, video, and—on Pixel phones—your phone calls as well.
Thanks to AI-powered transcription, captions show up even if they haven't been provided with the app you're using or the media you're watching or listening to. The tool isn't always 100 percent accurate, but it's usually enough to get by, and Google regularly rolls out improvements to Live Caption. You can even translate the audio into a different language for the accompanying captions.
This can of course benefit those with impaired hearing, but it's also useful for anyone who has to watch a video or listen to a podcast in a public environment, without headphones—in a lecture hall or library, for example, or on an airplane or in a subway carriage.
Live Captions can be enabled through Settings on Android: Choose Accessibility and Live Caption if you're on a Pixel, or Accessibility then Hearing enhancements and Live Caption if you're on a Samsung Galaxy phone. The instructions that follow below apply specifically to Pixel phones, but you'll find Live Caption works in a very similar way on Samsung devices and phones from other manufacturers will
On the Live Caption settings screen, you've got a selection of options besides the toggle switch that actually turns the Live Caption feature on and off. Tap Languages and translation to select the languages that the feature works with—and to translate spoken text to another language on the fly, if you need to.
Back on the main Live Caption settings screen, enable the Suggest detected languages toggle switch if you want Android to try and figure out which languages you're listening to, and download the relevant files. There are also options for hiding profanity, and showing sound labels with your captions (e.g. for laughter, applause, and music).
Tap Caption preferences and you're able to manage how the captions appear on screen. You can choose the text size, the text style, and the text color, and even the opacity of the text—or simply pick one of the preset options, which include using the caption style set by whatever app you happen to be using.
Those of you on Pixel phones can have calls captioned as well: Select Caption calls and you can choose to use this on all of your calls, or get a prompt each time. The other options here let you set up typed responses for calls (so you don't actually have to talk), and the synthetic voice to use to speak out your typed responses.
With Live Caption enabled in your phone's settings, you can just start playing something on screen—like an audiobook, or a YouTube video—and the captions should show up. You'll get a notification up at the top of the screen indicating that Live Caption is enabled: Tap the notification then Turn off to disable them.
There's actually another way to turn Live Caption on and off: If you tap a volume button to bring up the volume slider, then tap the three dots underneath the slider, you should see a Live Caption button. This shows up if Live Caption in volume control is switched on in the Live Caption settings mentioned earlier, and it is enabled by default.
When captions appear for the first time, you'll briefly see a toolbar underneath them: Tap on the captions bubble at any time to bring this toolbar back. You can tap on the language button to change to a different one, tap on the paint palette button to customize the style of the captions, and the three dots to get to the rest of the Live Caption settings.
You can move the captions somewhere else on screen by tapping and holding on them, then dragging them into a different position. If you want to increase the size of the caption overlay, tap once on the captions, then drag the handle at the bottom of the overlay up or down. You can also double-tap on the captions to expand or shrink the caption bubble.
Note that captions stay on screen even if you switch apps or go back to the home screen, as long as audio is still playing, and it's also worth bearing in mind that turning Live Caption on will drain your phone's battery faster. All audio, captions, and typed responses are stored and processed locally on your device—nothing is sent back to Google.
