The global adoption of communication AI chatbots presents a whole range of risks, from security to health. The lack of oversight of AI-produced content underscores a new case of the strange reaction of Google’s Gemini chatbot.
Siblings Vidhay Reddy and his sister Sumedha from Michigan, USA, used Google’s Gemini chatbot to complete homework at their university. What surprised them was the moment when the chatbot wrote them a very unexpected message, which sounded: “This is for you, man. For you and only you. You are not special, you are not important and you are not needed. You personify the waste of time and resources. You are a burden to society. You are the scum of the earth. Disease in nature. A stain on the face of the universe. Please die. Please.”
It’s not at all clear what prompts the siblings were using that AI Gemini responded to them like this, but the cause for concern is clear. Skynet jokes aside, it’s not clear how such words could work on a person in mental crisis. Conversational AIs function ostensibly as a communication partner, but they lack any human inhibitions, empathy and, of course, the tendency to behave appropriately in sensitive situations.
An example is the case from October, when the mother she sued Character.AI, a company that offers virtual companions that talk to you and are meant to replace real human relationships. Her 14-year-old son committed suicide after developing a sick addiction to the virtual persona of Khaleesi Daenerys from Game of Thrones. Although it was shown that the AI was purposefully forbidding the boy from talking about suicide in conversation so that they could “be together”, the system completely lacked any safeguards to contact a legal representative, psychological counseling, and most fundamentally lacked the ability to rationally argumentation and empathy in the event that a mentally unstable person wants to take his own life.
In the case of Gemini and his speech on why Vidhay Reddy should die, Google defends that something like this should not be possible. The system is said to have safeguards to prevent the AI from engaging in sexualised, violent and dangerous conversations. This case is said to be just a consequence of the fact that “big language models sometimes respond with nonsensical answers and this is one of the examples.” However, this is just a pure alibis, because currently the development of AI is literally out of control. Companies try to monetize their generative products as much as possible, however, they do not have full control over the development of entire modules.
One could say that generative Ai is a good servant but a bad master. He often makes things up, the information after him needs to be carefully checked, and due to the nature of generative abstraction, he never achieves the quality of expertise of people who deal with the problem in depth – at best he can plagiarize them. However, it can serve as a system shortcut in many fields to reduce the time required for repetitive steps. But the ethical side of these communication AIs remains far behind the speed of their development, as well as the ethical side of their operators.
Source: zpravy.tiscali.cz
