AI models often give the right answers but point to the wrong sources

Research from Peking University reveals that AI models like GPT and Gemini often cite incorrect text passages that do not support their answers. This phenomenon, termed "attribution hallucination," poses risks in regulated fields such as law and medicine. The newly introduced CiteVQA benchmark systematically tests for this issue, aiming to measure the accuracy of citations in document analyses. This benchmark emphasizes the critical need for reliable evidence in sensitive applications.