Over the years, artificial intelligence has been seen as a potential savior of the medical system, capable of revolutionizing the diagnosis and treatment of patients.
However, a recent study published in The BMJa leading medical publication, comes with an unexpected insight: AI models may suffer from cognitive impairments similar to those of early-stage dementia in humans.
Cognitive tests and artificial intelligence
The researchers evaluated the most advanced AI models, such as ChatGPT 4 and 4o, Claude 3.5 and Gemini 1 and 1.5, using the Montreal Cognitive Assessment (MoCA) test.
It is typically used to detect cognitive impairment in elderly patients and measures skills such as attention, memory, language, and visuospatial functions. The maximum score is 30 points, and a result of at least 26 is considered normal.
The results were interesting: ChatGPT 4o got the best score, 26/30, followed by other models that came close, but did not reach the normal threshold. In contrast, Gemini 1 scored only 16 points, suggesting serious limitations in certain areas.
Where does artificial intelligence “fail”?
Visual-spatial tests and executive functions proved to be the hardest for the AI. For example, tasks such as drawing a clock to show the time 10:11 or connecting letters and numbers in ascending order were completed unsatisfactorily.
Although the models performed well on tasks involving attention, language and basic reasoning, they had difficulty on delayed recall tests and did not demonstrate empathy.
For example, when shown an image of a child about to fall, neither model was able to identify the situation as dangerous, a detail that a human doctor would have noticed on the spot.
What does this mean for the future of AI in medicine?
Although AI models excel at diagnosing medical conditions, their limitations in understanding complex situations and demonstrating empathy are clearly major barriers.
The study’s authors point out that impairments in executive and visuospatial functions could prevent the widespread use of AI in critical medical settings.
Moreover, the poorer performance of older versions of these models raises an interesting analogy: AI could experience a kind of “cognitive decline” as time passes or the technology ages.
Artificial Intelligence: Doctor or Patient?
The researchers’ conclusion is as ironic as it is serious: far from replacing neurologists, AI models could become “virtual patients” who show signs of cognitive decline.
Source: www.go4it.ro
