AI chatbots reading X-rays can be dangerously confident even when they're wrong

The RadLE 2.0 benchmark assesses AI models in radiology on their ability to determine when to defer a diagnosis to human experts. Many models exhibit high confidence in incorrect findings, underscoring the gap between AI and human radiologists. Studies indicate that human radiologists still outperform AI models in diagnostic accuracy. It is crucial for AI systems to learn when to refrain from making a diagnosis, highlighting the need for enhanced learning mechanisms.