Fake news and real news: The distinction remains imperfect

How convincing is fabricated news compared with real news—and does AI production matter? Ádám Stefkovics and Dömötör Gere investigate this in a preregistered PNAS Nexus study. The original publication is linked in the references. [1]

The online survey covered 27 European countries and 27,227 participants; weighted analyses included 26,437. Each participant rated eight short news items about the Russo-Ukrainian war, combining real and fabricated content produced by humans or GPT-4o. Participants were not told the production source beforehand. [1]

Real news performed better, but the gap remained moderate: Mean credibility ratings were 2.60 for real news and 2.15 for fabricated news—a difference of 0.45 on a four-point scale. For willingness to share, the authors report an even smaller gap of around 0.18 points. Meanwhile, 57% of real items and 34% of fabricated items were rated mostly or completely real. Participants therefore made a discernible but far from reliable distinction. [1]

Differences between human- and AI-generated content were much smaller. This does not directly measure the ability to detect AI writing: participants rated veracity and sharing intentions, rather than identifying authorship. [1]

Important limitations apply: the study used selected items about one war-related topic and nonprobability online panels. Sharing intentions also differ from observed sharing on social networks. The findings cannot simply be generalized to every topic or everyday setting. [1]

Read as an indication, the study sharpens the challenge of evaluating information: plausible presentation alone is not a reliable sign of truth. As AI makes convincing content easier to produce, verifiable sources, context and critical comparison become increasingly important.