Scientist teaches AI to admit when it doesn't know the answer

Peter Behroozi, an associate professor at the University of Arizona’s Steward Observatory, has created a method to help AI systems identify their own limitations. His work addresses the issue of 'hallucinations', where AI confidently produces false information. This problem can have real-world consequences, from medical misdiagnoses to unfair housing decisions. Behroozi’s breakthrough did not originate from AI research but from his work on the 'Universe Machine', a project simulating galaxy formation. He adapted ray tracing, a film industry technique for realistic lighting, to function in billions of dimensions for AI models.

His approach uses Bayesian sampling, a statistical method once deemed too demanding for large neural networks. The technique enables AI to recognise uncertainty, effectively developing an awareness of its knowledge gaps. Examples of AI hallucinations include inventing fake facts, research papers, or books to support incorrect answers.

The findings have been published on the open-access platform arXiv. The associated code is now available for researchers worldwide to integrate into their own projects. Behroozi’s method allows AI systems to acknowledge when they lack sufficient information. The open-access publication and code release mean other scientists can build on this work. This could reduce the risks of false-but-confident outputs in critical applications.