One of the most useful features coming to Android Q makes it easier for the hard of hearing or deaf users to understand what’s happening in video and audio clips they’re watching or listening to. Live Caption will let you see any audio or video clip transcribed in real time, which The Verge’s Chris Welch is said to be done “with extremely accurate results.” The Live Caption is overlaid on top of whatever media app you’re using. It even works for video chat apps like Skype and Google Duo as well as with video or audio you record yourself. It makes use of on-device machine learning so it doesn’t have to send your data to the cloud to interpret what’s happening. It also works even when volume is turned down or muted. However, it won’t save the translations for later use.
The captions, which appear in a black box that you can move around, disappear when the content is over. It is turned off by default, but you can enable the feature in your phone’s accessibility settings. These can then be enabled through a software icon that appears in the volume UI pop-up. The feature will prove to be useful even to those without hearing problems. Imagine being on a crowded train and just wanting to watch a clip without having to turn up the volume to hear what’s happening or use it as a guide for transcribing.