These content creators do not exist
In which Facebook's recommendation algorithm serves as the gateway to a network of inauthentic accounts
Meet Stephanie Daltoso, Preston Lanier, Mark Siddle, Louis Rodriguez, Seam Chakriya, Mavin Jack, Ly Meng, Leticia Gamboa, and 30 of their closest friends, a coterie of Facebook acquaintances who, for better or worse, do not actually exist. These Facebook accounts, which variously present themselves as “content creators”, “digital creators”, and “video creators”, all have StyleGAN-generated faces as their profile photos. At least some of their content, which includes a mix of plagiarized photographs and AI-generated images produced by text-to-image models such as Stable Diffusion, is being algorithmically recommended by Facebook to new users.
Most of these alleged content creators have multiple other content creators with GAN-generated faces among their friends and followers. By starting with one account and recursively exploring the friend and follow relationships, one can map out this network of inauthentic accounts. This process yields a set of 38 Facebook accounts with GAN-generated faces, most of which claim to be based in either New York or Los Angeles. (There are also several “content creator” accounts with real photos rather than GAN-generated faces lurking in the friend lists of the accounts with GAN-generated faces. It is possible that these accounts are likewise inauthentic, but they are excluded from the present analysis.)
The majority of the GAN-generated faces used by this network have male appearances, which is presumably a deliberate choice on the part of the network operators, as commonly available tools for generating these faces produce male and female faces at roughly the same rate. The images contain multiple anomalies indicative of their synthetic origin, including surreal backgrounds on several images, an incorrectly-shaped hand on Lexi Jones’s image, and identical eye placement on every single image (a fingerprint of all unmodified StyleGAN-generated faces). In addition to using the GAN-generated faces as profile photos, many of the accounts also have them set as their banner images.
In an interesting twist, some of the accounts in the network claim to be in relationships with other accounts in the network. Two pairs — Lexi Jones and Alan Newman, and Ly Meng and Amari Larson — have allegedly been together since January 2023, per their Facebook relationship statuses. To date, the operators of the network have made no effort to embellish this aspect of the accounts with additional posts chronicling the development of these GAN-tastic romances.
The content posted by these accounts is a mix of AI-generated images, real photographs harvested from the Internet, and maps (generally centered on the New York or Los Angeles areas). Some of the AI-generated images, such as the house in the post on the left in the above collage, have racked up significant numbers of likes, comments, and shares, and are currently being algorithmically recommended by Facebook. I was shown AI image posts from the Jelani Stewart account both on a newly created Facebook account and a longstanding sock account that I do not post with.
Another great job documenting this issue. FB has had problems with this for a very long time. It ruins the experience and I wonder why they don’t really address these bot accounts with more robust account creation tools. Possibly it helps to inflate their user base with false accounts so they can somehow say they have more users than they really do. There is always some scam going on with most of them. Starts out with Hi/Hello and then eventually goes straight for crypto scams, needs for money etc.