AI art and T-shirt spam
Analysis of a network of inauthentic X accounts that posts AI-generated images and spams popular political accounts with ads for T-shirts
The use of generative AI on social media has become increasingly common over the last few years. While some uses of this technology have been dramatic, such as an AI-generated image of an explosion at the Pentagon that managed to briefly disrupt the real-world stock market, or bogus “journalists” with AI-generated faces pretending to cover major wars, others are more mundane, such as low-effort attempts to sell merchandise. Here’s a look at a network of 24 mutually interacting X accounts that post AI-generated art images and spam popular influencer accounts with T-shirt ads.
Twelve of the twenty-four accounts in the network have female names and avatars, and were created fairly recently, in either August or September 2023. These twelve accounts regularly post AI-generated art images, which are generally reposted by several of the other accounts in the network. Despite the fact that one can generate a nearly unlimited set of unique images with text-to-image AI models, these accounts frequently post the same image multiple times. Some of these images are obviously stylized or cartoonish; the more photorealistic images contain occasional artifacts indicative of their artificial origin, such as an extra finger on a chef’s right hand.
The other twelve accounts in the network have male names and are substantially older than the female-presenting accounts, with creation dates ranging from April 2009 to September 2013. These accounts do not post AI-generated images; instead, several of them post real (plagiarized) photographs, and as with the AI-generated images posted by the recently created accounts, the photographs are often repeated. The photographs generally depict outdoor scenes, such as beaches or closeups of wildlife, and the posts containing photographs are often reposted by other accounts in the network.
The accounts in this network, both old and new, only have a few dozen followers each. Even this minuscule degree of popularity is somewhat inflated, however, as every account in the network is followed by most or all of the other accounts in the network. The remainder of the accounts’ followers seem to be mostly random legitimate users who were followed by the fake accounts and followed them back.
In addition to following one another, the accounts in this spam network also frequently repost each other’s posts, with each account having been amplified by other members of the network at least 469 times. The accounts also sometimes repost content from larger accounts that are not part of the network. The out-of-network content is mostly photos and videos of cute cats, AI-generated art, and material related to Taylor Swift.
The accounts in this network also post replies, and these replies provide a clue as to the network’s potential purpose: namely, T-shirt sales. The network’s replies generally consist of a random word accompanied by a link to an obscure website hawking T-shirts. Most of these replies are to large liberal (or conservative anti-Trump) influencer accounts such as @RonFilipkowski, @OccupyDemocrats, and @RpsAgainstTrump, although the network also replies to AI art and Taylor Swift-themed accounts, including Swift’s official @taylorswift13 account.
Although many of the accounts in this network use photographs of real people as profile images, these images are stock photos and are therefore highly unlikely to depict the account operator. TinEye reverse image search was effective at tracking down prior uses of these photographs on the Internet (and running the profile photos of unfamiliar social media accounts through reverse image search tools is a good habit to get into in general). A few of the accounts use drawings or AI-generated art as profile images instead of plagiarized photographs.
Great work. If someone wanted to proactively hunt for these kind of spam networks, what would you suggest they do?
I sense another purpose for them. T-shirts aren’t exactly a thing these days. If you want one, you can easily get one at any mall kiosk or full store complete with any graphics you want on them. At end of day, these are always built and used for nefarious purposes. Great work as always. Without you people would not know a quarter of what’s really going on out there.