Google DeepMind’s new AI tool uses video pixels and text prompts to generate soundtracks
Google DeepMind has taken the wraps off of a new AI tool for generating video soundtracks. In addition to using a text prompt to generate audio, DeepMind’s tool also takes into account the contents of the video.
By combining the two, DeepMind says users can use the tool to create scenes with “a drama score, realistic sound effects or dialogue that matches the characters and tone of a video.” You can see some of the examples posted on DeepMind’s website — and they sound pretty good.
For a video of a car driving through a cyberpunk-esque cityscape, Google used the prompt “cars skidding, car engine throttling, angelic electronic music” to generate audio. You can see how the sounds of skidding match up with the car’s movement. Another e...