Studies Show People Can’t Tell AI Art From Human Art

Featured image for Studies Show People Can’t Tell AI Art From Human Art

Earlier this month, an anonymous conceptual artist called SHL0MS posted an image on X with a simple prompt: “I just generated an image in the style of a Monet painting using AI. Please describe, in as much detail as possible, what makes this inferior to a real Monet painting.”

The post carried X’s “Made with AI” label. 6.7 million people saw it. Thousands obliged with detailed, confident critiques. The brushstrokes were “extremely fake.” The color choices showed “no cohesion to the depth.” The reflection of the tree bled “into the lilypads with no regard for spatial depth or contrast.” One critic called it an “incoherent muddle of inconsistently saturated greens.” Another wrote an 850-word breakdown of its shortcomings. Someone else offered a more concise assessment: “It looks like sh*t, and is sh*t.”

Two days later, SHL0MS revealed the image was a genuine Monet.

Specifically, a painting from the Water Lilies series, painted around 1914 and housed in the Neue Pinakothek in Munich. The signature was cropped out, but everything else was original.

Studies prove people can’t tell the difference

What this social media stunt demonstrates casually, scientists have verified experimentally — people cannot tell when something is AI-generated. A 2025 survey by Conjointly asked 301 U.S. adults to identify which images were real photographs and which were AI-generated. Real images were correctly identified 49% of the time. AI images were correctly identified 52% of the time, indistinguishable from random guessing.

What’s worse is, people don’t realize they can’t. 42% of respondents reported feeling “confident or extremely confident” in their ability to tell the difference, despite doing no better than a coin flip.

These results hold up across larger and more rigorous studies. Ha et al. (2024) tested participants at varying levels of artistic expertise on images from five different AI generators. Among non-artists, accuracy was 59%, with false positive and false negative rates both above 40%. The researchers’ conclusion was direct: “General, non-artist users are unable to tell the difference between human art and AI-generated images.”

The pattern extends beyond images. Jakesch, Hancock, and Naaman (2023) ran six experiments with 4,600 participants evaluating AI-generated and human-written text across professional, hospitality, and dating contexts. Accuracy remained a coin flip. The researchers tried offering monetary incentives for correct answers. No improvement. They tried giving participants immediate feedback on whether their guesses were right. Still no improvement.

A separate study on GPT-3 text, Clark et al. (2021), found exactly 50% detection accuracy across stories, news articles, and recipes. Mind you, GPT-3 is so old that it was out of date when ChatGPT launched. Detection has been a coin flip for as long as AI chatbots have been with us.

Inability to distinguish quickly becomes bias

But detection is only half the problem. In a 2024 study published in Scientific Reports, Porter and Machery asked over 1,600 participants to identify which poems were written by humans (Shakespeare, Dickinson, Plath, and others) and which were generated by ChatGPT. When the researchers had a separate group rate the poems on quality without being told to guess the source, AI-generated poems scored higher on 13 of 14 quality dimensions, including beauty, profundity, and meaningfulness.

A similar finding appeared in Grassini and Koivisto (2024), who showed 201 participants a mix of AI-generated and human-created images across five art styles. When participants didn’t know the source, AI images were rated significantly higher on liking and positive emotional response. The effect was large. But that preference disappeared, and in some cases reversed, when participants believed the images were AI-generated.

Bellaiche et al. (2023) studied this phenomenon directly. In their study, every single image shown to participants was AI-generated. There were no human-created works in the experiment at all. Participants were simply told, at random, that some images were “human-created” and others were “AI-created.” The images labeled as human-created were rated higher on every aesthetic dimension: liking, beauty, profundity, and worth. The bias was weakest for surface-level judgments like liking and strongest for deeper evaluative criteria. Art labeled as AI-created was rated as less profound and less worthy consistently.

Horton, White, and Iyengar (2023), in a series of six experiments with nearly 3,000 participants published in Scientific Reports, found the same thing from a different angle. Over 70% of their participants reported that the AI-generated and human-created images were visually indistinguishable. They said they couldn’t tell the difference. And then they rated the AI-labeled works lower anyway, across dimensions including liking, skill, and expensiveness. In one experiment, the same artwork lost 62% of its perceived monetary value when labeled as AI-generated rather than human-made. The researchers put it plainly: “People devalue art labeled as AI-made across a variety of dimensions, even when they report it is indistinguishable from human-made art.”

A 2025 meta-analysis by de Rooij, synthesizing 191 effect sizes from studies spanning 2017 to 2024, confirmed the overall pattern. The negative bias against AI-labeled art was present across nearly every study, but its strength varied by what was being judged. Surface-level aesthetic reactions (does it look nice?) showed a small effect. Emotional responses (does it move you?) showed a small-to-moderate effect. Judgments about meaning, profundity, and creativity showed the largest bias. The further a judgment moved from “how does this look” toward “what does this mean,” the more the AI label degraded the evaluation.

The meta-analysis also found the bias was stronger in online studies than in laboratory or real-world gallery settings, and stronger when AI was framed as an autonomous creator rather than as a tool in a human creative process — suggesting the strength of the bias partly tracks how AI’s role is described, not just whether it was involved.

Who is the most biased against AI art?

Grassini and Koivisto (2024) measured personality traits and attitudes alongside their art evaluation study. The strongest predictor of negative bias toward AI art was creative personal identity — how strongly a person identifies as creative.

In their defense, professional artists do have a better-than-average identification rate. Oil painter Kendric Tonn and art historian A.V. Marraccini were the only two people in the SHL0MS thread who correctly identified the Monet, and 75% of professional artists in the Ha et al. study correctly identified the art’s origin. (But even their accuracy dropped to 66% for oil and acrylic styles).

Technology attitudes were the next-strongest predictor: people with positive attitudes toward technology predictably showed substantially less bias, while people with negative attitudes showed more. Age mattered too, with older participants devaluing perceived-AI images more strongly.

In closing

None of this is to say there are no legitimate concerns about AI-generated art. Artists’ compensation and the dangers of emotionally bonding with non-human systems are criticisms that still loom large. But those are different arguments from the one most commonly made, which is that AI work is somehow demonstrably worse in quality. To that, one can only say that the evidence shows otherwise.

AI was used in the writing of this article.

Enjoy this kind of writing?

I write about AI, intentional living, and doing meaningful work in a world that won't stop changing. Join 700 other subscribers.