The reliability of the answers given by AI chatbots has been the subject of debate since the explosion started by ChatGPT at the end of 2022. All of them include safety warnings indicating that they can make mistakes, but They are used as if they did not exist because of the comfort they offer. However, these errors can be critical when discussing important topics such as those related to health. According to an investigation by The Guardian, summaries generated by AI (AI Overviews, called Views created by AI in Spain) hide security warnings in medical consultations until the user clicks the ‘Show more’ button. That is, they do not appear in the initial response that Google gives.
Not only that, but when they are displayed after pressing the button, they appear below the medical information and in smaller fontwhich makes it easier for them to go unnoticed.
‘This is for informational purposes only. For medical advice or a diagnosis, consult a professional. AI responses may include errors‘, a user can read after the entire Google Search response. If it has gotten there.
To questions from the British media, Google explains that ‘it is inaccurate to suggest that AI Summaries do not encourage people to seek professional medical advice’ and that these ‘will inform people when it is important to seek expert advice or verify the information presented’, ‘encourage people to seek professional medical advice’ and that they frequently mention within the summary itself the advisability of seeking medical attention ‘when appropriate’.
But that ‘when appropriate’ may be far from what is necessary, according to experts consulted by the media. Pat Pataranutapornassistant professor, technologist, researcher in the Massachusetts Institute of Technology (MIT) and world-renowned expert in AI, states that ‘The absence of warnings when users are initially served medical information creates several critical dangers. First, even the most advanced AI models today still they hallucinate misinformation or show complacent behavior, prioritizing user satisfaction over accuracy. In healthcare contexts, this can be really dangerous. Second, the problem is not just the limitations of AI: there is also the human side of the equation. Users can not provide all the necessary context or may ask the wrong questions by misinterpreting their symptoms. Warnings act as a crucial point of intervention. They break that automatic trust and push users to relate more critically to the information they receive.’
For its part, Gina Neffresponsible AI professor at the Queen Mary University of Londonnotes that the ‘problem of bad AI Summaries is intentional. AI Summaries are designed for speed, not accuracyand that leads to errors in health information, which can be dangerous.’
This research follows another published by The Guardian in January in which it showed that users were exposed to risks from receiving false and misleading health information in AI responses in Search. On these, Neff argues why it is essential to keep security warnings visible to the user: ‘Google makes people have to click before finding any warning. Speed readers may think the information they get from AI Summaries is better than it is, but we know you can make serious mistakes‘.
Following this information, Google removed AI-generated responses for some medical queries.
Sonali Sharmaresearcher at the Center for AI in Medicine and Imaging of the Stanford University (AIMI), tells The Guardian that ‘the main problem is that these Google AI Summaries appear at the top of the search page and often offer what seems like a complete answer to the user’s question at a time when people are trying to access information and get an answer as quickly as possible. For many people, because that single summary appears immediately, it basically creates a sense of calm that discourages further searching, or scrolling through the full summary and clicking ‘Show more’, where a notice might appear. What I think can cause real-world harm is the fact that AI Summaries can often contain partially correct and partially incorrect information, and it becomes very difficult to distinguish what is accurate and what is notunless you are already familiar with the topic.’
Tom Bishoppatient information officer at Anthony Nolana blood cancer charity, amends Google: ‘We know that misinformation is a real problem, but when it comes to health misinformation, it is potentially very dangerous. That notice needs to be much more visible, just to make people stop and think… ‘Is this something I need to check with my medical team instead of acting on it?’ Can I accept this as is or do I really need to look into it in more detail and see how it relates to my specific medical situation?’ Because that’s the key here. I would like that notice to be right at the top. I would like it to be the first thing you see. And ideally, it would be the same font size as everything else you’re seeing there, not something small and easy to overlook.‘.