Civitai, an online marketplace for sharing AI models, just introduced a new feature called “bounties” to encourage its community to develop passable deepfakes of real people, as originally reported by 404 Media. Whoever concocts the best AI model gets a virtual currency called “Buzz” that users can buy with actual money.
Many of the bounties posted to the site ask users to recreate the likeness of celebrities and social media influencers, most of them female. The reporting also calls out the lion’s share of these results as “nonconsensual sexual images.” This is the kind of thing that has been proliferating across the internet for years and years, but artificial intelligence allows for a more realistic end result. Additionally, 404 Media found some requests for private people with no significant online presence, making this even more creepy.
“I am very afraid of what this can become,” Michele Alves, an Instagram influencer who has a bounty on Civitai, told 404 Media. “I don't know what measures I could take, since the internet seems like a place out of control.”
According to market firm Andreessen Horowitz, Civitai is the seventh most popular generative AI platform at the moment. In other words, there are a whole lot of eyeballs on these bounty requests. It only took staffers at 404 Media moments to source images sent via a bounty request to a private person with personal social media accounts boasting just a few followers. The person who posted the bounty claimed it was his wife, but her social media accounts said otherwise. Gross.
One Civitai user declined the bounty on the grounds that it was “asking for legal problems in the future.” To that end, Virginia just updated its revenge porn laws to punish deekfake creators with up to one year in jail. Still, this particular request was fulfilled and several images were uploaded to the site, though they were non-sexual in nature.
It’s worth noting that very few of the bounty requests specifically state the poster’s looking for sexual material, couching the request in vague language. Some, however, go all-in, using terms like “degenerate request” along with comments on female breast size. Civitai, for its part, says that these bounties should not be used to create non-consensual AI-generated sexual images of real people.
However, both sexual images of public-facing figures and non-sexual images of regular people are allowed. After that, it’s just a matter of combining the two. 404 Media used the company's text-to-image tool to create non-consensual sexual images of a real person “in seconds.”
This article originally appeared on Engadget at https://www.engadget.com/popular-ai-platform-introduces-rewards-system-to-encourage-deepfakes-of-real-people-194326312.html?src=rss