Quantum simulators plagued by silent errors and undetected bugs, study reveals

A new study has uncovered significant reliability issues in quantum simulation software. Researchers analysed 394 confirmed bugs across 12 widely used open-source quantum simulators. Their findings challenge the common belief that simulator results provide a dependable benchmark for validating quantum algorithms. The investigation focused on bugs confirmed through merged pull requests, ensuring each issue had a verified fix. Of the total, around 40% were quantum-specific failures tied to modelling quantum phenomena or implementing gates and circuits. Another 63 bugs originated from classical infrastructure problems, such as memory management, proving that even non-quantum components play a critical role as systems grow more complex.

Silent errors posed a particular concern, with roughly 100 bugs producing incorrect yet seemingly valid outputs. These failures did not trigger crashes or warnings, making them difficult to detect. The study also revealed that most errors were caught through user reports rather than automated testing, suggesting current quality assurance methods are inadequate. Researchers emphasised that many logical correctness issues slip through existing tests. This gap highlights the need for stronger automated validation tools to ensure quantum simulators deliver accurate and trustworthy results.

The findings underscore weaknesses in both quantum and classical components of simulation software. With silent errors and undetected failures common, the study calls for improved automated testing to strengthen reliability. More robust validation will be essential as quantum computing applications expand in research and industry.