GAN-tastic catfishing
When social media accounts with artificially generated faces build entirely fictional lives on Twitter, they are sometimes widely accepted as real people
Some inauthentic social media accounts have an obvious motive behind their existence, such as manufacturing credibility as a fake journalist or public figure, astroturfing discussion of a viewpoint, topic, or person, or plain old financial scamming. In other cases, however, no political or financial motive is apparent, and maintaining the fake persona and drawing people into the illusion seems to be an end in itself (a form of catfishing). The more successful of these fake accounts manage to build authentic audiences that accept them as real people and become emotionally invested in the accounts’ imaginary lives. This article focuses on two moderately popular (and presently suspended) inauthentic personas from Twitter/X: “Paul Fox” (@PaulFox50854324), and “Gabby” (@Gabby_UCMaroon, @GabbyRedux, and @gabby_ucm), both of which used StyleGAN-generated face images as their avatars and employed plagiarized photos to embellish their fictitious online lives.
The “Paul Fox” account (@PaulFox50854324) was created in May 2019 and banned by X/Twitter in late June 2023, after a span of slightly over four years. In addition to using a GAN-generated face as a profile image, this account also misrepresented a GAN-generated face as a neighbor’s child in a June 18th, 2023 Father’s Day post. The Father’s Day post in question racked up over 800 reposts in roughly 36 hours, and received hundreds of positive replies from users who, by and large, seemed to believe that the image was a real photograph of a child.
The operator of the “Paul Fox” account used stolen images to help fabricate the account’s life story. Items of property such as a walled garden, a glass of grape Kool-Aid, a laundry-bedecked treadmill, and a 1966 Corvette convertible that “Paul Fox” both did and did not give to his eldest son as a gift are all photographs copied from elsewhere on the internet. Even @PaulFox50854324’s son Evan and his neighbor’s dog Hans are plagiarized photos, easily detected via TinEye reverse image search.
The “Gabby” persona appears to have first surfaced in January 2021 as @Gabby_UCMaroon. The operator created a backup account (@GabbyRedux) in April 2021. Both of these accounts were banned during or before September 2021, at which time they were replaced by @gabby_ucm, which was in turn suspended at the end of January 2022 (a development which apparently annoyed some people). Both the @Gabby_UCMaroon and @gabby_ucm accounts used the same GAN-generated face as their avatar, while the @GabbyRedux backup account used a stolen photo.
The GAN-generated face was not the only image that the “Gabby” persona misrepresented as a photograph of the account operator. In January 2021, not long after being created, the @Gabby_UCMaroon account posted an image of a woman standing next to a window and claimed in replies that the photo was an image of the person running the account. This was false, however — TinEye reverse image searches reveal that the photograph in question had been floating around the web for quite some time.
In December 2021, “Gabby” posted another alleged photo of the individual behind the “Gabby” accounts, this time on the @gabby_ucm account. Although “Gabby” claimed that the photo was taken “recently”, it was originally posted in 2019 on an Instagram page belonging to a hair salon in Japan, a country to which “Gabby” claimed never to have traveled. In addition to plagiarizing the image, the operator of the “Gabby” account made some modifications to the facial expression and features before posting it.
Both the “Gabby” and “Paul Fox” accounts engaged frequently with their followers, and both had a habit of replying in ways that reinforced the impression that the GAN-generated faces used by the accounts were real photos of real people. This was sometimes done by adding additional background details (“we attend the same church, we’re all well-known to each other”) and sometimes by accepting appearance-based personal compliments directed at the artificially-generated faces. For better or worse, these synthetic images are of sufficient quality to be accepted as actual humans by unsuspecting users who encounter them in the wild. In fact, recent research suggests that people are more likely to accept a GAN-generated face as “real” than a photograph of an actual human.
This problem will only worsen as synthetic media technology improves. Although there’s no “magic bullet” solution to this issue, a good starting point would be for social media platforms to require users to disclose the use of synthetic images, especially when such images are used in a context where they are likely to be mistaken for real photographs. On the user side of things, particularly in the realm of politics, people will need to not only adapt to an ever-changing landscape of synthetic media, but also (and perhaps more importantly) to embrace a greater willingness to call out deceptive content posted by members of one’s own “side”.