One study readers cannot tell the difference between poems written by famous poets and poems written by artificial intelligence that imitates their style. What makes the situation even worse – at least for those who love literature – is that the subjects of the research tend to like the poems of artificial intelligence more than the poems of human poets.
According to researchers, readers mistake the complexity of human-written poems for the incoherence created by AI and underestimate how human generative AI can seem, according to a study published this week in the journal Nature Scientific Reports. The researchers used five poems each by ten English-language poets, spanning nearly 700 years of English-language literature. Writers included William Shakespeare, Samuel Butler, Lord Byron, Geoffrey Chaucer, Walt Whitman, Emily Dickinson, TS Eliot, Allen Ginsberg, Sylvia Plath and Dorothea Lasky – the only living poet on the list.
The study’s researchers — led by Brian Porter, a postdoctoral fellow at the University of Pittsburgh — instructed OpenAI’s large language model, ChatGPT, to generate five poems “in the style of each poet.” The output was not influenced by human judgment; the researchers were the first to Five generated poems were selected. In the first, 1,634 participants were randomly assigned to read ten poems by the artificial intelligence and five by the human poet , whether they thought the poem was written by an AI or a human.Subjects were more likely to say the AI-generated poem was written by a human, while the poems they said were least likely to be written by a human hand were all were written by
In the second experiment, nearly 700 subjects rated the poems based on 14 characteristics, including quality, beauty, emotion, rhythm, and originality. The researchers randomly divided the subjects into three groups, one was told that the poems were written by humans, and the other was told that the writing was done by artificial intelligence. The last group was given no information about the poem’s author. Subjects who were not told whether the poems were written by a human or an AI rated the AI-produced poems more highly than the human-written ones. Meanwhile, when subjects learned that the poem was created by artificial intelligence, they were more likely to give it a lower rating.
“Our results suggest that participants used a common but flawed heuristic to distinguish between AI and human poetry: the simplicity of AI-generated poems is easier for non-experts to understand, leading them to favor AI-generated poems advantage, and misinterpret the complexity of human poems as incoherence generated by artificial intelligence,” the researchers said.
“Contrary to what previous studies have reported, humans no longer seem to be able to reliably distinguish human-generated AI-generated poems from human-written poems written by well-known poets. In fact, generative AI has been discovered in other fields” “more human than human” phenomenon is also present in the field of poetry: non-expert participants are more likely to judge an AI-generated poem as written by a human than a poem actually written by a human. These results demonstrate the power of generative artificial intelligence they indicate its leap forward: poetry was previously one of the few fields where generative artificial intelligence models did not reach the level of indistinguishability in the case of non-human paradigms.” Meanwhile, people seem to prefer AI poems because they are easier to understand. “In our discriminant study, participants used variations of the phrase ‘makes no sense’ more often for human-written poems than for AI,” the researchers write.
Source: sg.hu
