404 Media has been pulling the thread of the great mystery of "What the heck is going on with all this AI image spam on Facebook?" Over the last few months, they've reported on various versions of AI people doing chainsaw carvings, children showing off bicycles made of vegetables, old people blowing out birthday cakes, dying or mutilated children, and my personal favorite: Shrimp Jesus.
Finally, Jason Koebler presents his theory of what all this AI engagement bait spam means:
I think we should not view Facebook's AI spam through the lens of the "Dead Internet." The platform has become something worse than bots talking to bots. It is bots talking to bots, bots talking to bots at the direction of humans, humans talking to humans, humans talking to bots, humans arguing about a fake thing made by a bot, humans talking to no one without knowing it, hijacked human accounts turned into bots, humans worried that the other humans they're talking to are bots, hybrid human/bot accounts, the end of a shared reality, and, at the center of all of this: One of the most valuable companies on the planet enabling this shitshow because its human executives and shareholders have too much money riding on the mass adoption of a reality-breaking technology to do anything about it.
My take: The weird and sometimes unsettling nature of AI images is fascinating, but underneath the phenomenon of these AI-powered pages proliferating, there appears to be a classic case of traditional engagement bait. The AI stuff builds up a page's following — and that can be profitable for the page owner.
The question I'm left with is why is Facebook allowing this? 404 Media's report notes that occasionally, Meta will take action against certain pages, but only in really specific cases — like with a hacked account and when it removed AI-generated images of disfigured children. Those probably ran afoul of some other existing content guidelines.
I have a few theories about why Facebook is (for now) allowing this kind of AI image spam:
It's an election year, and Facebook has bigger content-moderation issues to deal with.
The images themselves are mostly benign, and the individual images don't run afoul of content guidelines. Shrimp Jesus is absurd, and things like it may ruin the Facebook experience in the long run, but for now, it's not really an issue of violating the content rules.
Engagement bait isn't necessarily a banned tactic, either. Nothing says you can't post lots of engaging content to trick boomers (and others) into commenting. There are ways to violate the rules with certain spam tactics, but maybe these AI-fueled accounts aren't crossing the line.
Facebook will start to detect and label some AI-generated images, but there's no rule that says you can't post something just because it's AI-generated. (When I asked about how they're handling these, a rep for Meta pointed me to their announcement of this policy.)
(404 Media's theory): Meta is so invested in AI that it wants us to get used to seeing AI images, and so they're willing to let it go.
