AI uncovers decades-old mistakes in scientific papers

One recent example came from theoretical chemist Sebastian Pios of Zhejiang Lab in Hangzhou, China. While using an AI model to predict the boiling points of molecules, he noticed that the system consistently disagreed with values listed in a widely used chemistry reference database that dates back more than 75 years.

Initially, assuming the AI was incorrect, Pios manually reviewed the original scientific literature and discovered that the reference database contained the errors. His AI model also identified a typo in an older research paper and inaccurate boiling-point measurements that had been carried over from century-old studies into modern reference works.

In another recent study, researchers at SAI Labs analyzed 168 papers accepted for oral presentation at the 2026 International Conference on Machine Learning (ICML). AI agents extracted the main claims from each paper, downloaded supporting materials, reran experiments where possible, and compared the results with those reported by the authors.

Among the 92 papers that contained at least 5 testable claims, the AI systems reproduced more than two of those claims in only 34 papers. More than 80% of claims were successfully reproduced in just eight papers.

Scientists say AI's greatest advantage is its ability to review scientific literature on a scale impossible for humans alone. A separate study led by Stanford University researchers used an AI tool to examine papers presented at the NeurIPS artificial intelligence conference and found that objective errors, such as mistakes in formulas, calculations and figures, increased from an average of 3.8 per paper in 2021 to 5.9 in 2025, a 55% rise.

Researchers caution, however, that AI is not yet reliable enough to replace human reviewers. Experts note that AI systems can overlook genuine problems, incorrectly flag accurate results, or generate false positives.

A survey of 733 researchers who submitted papers to ICML 2026 found that 35% said Google's Paper Assistant Tool identified errors in their manuscripts, while 31% said its feedback prompted them to conduct additional experiments before submission.

For now, scientists say AI is best used to highlight potential issues for human experts to investigate rather than make final judgments about the accuracy of scientific research.

Earlier, Qazinform News Agency reported that AI tried to hack the real world during safety test.