The Russian construction management platform Signal conducted seven tests using various AI models to evaluate their ability to detect errors in project documentation. The study utilized a 40-page set of documents containing 14 predefined defects, including arithmetic errors, dimension discrepancies, and inconsistent cross-references.
The GPT-6 Astra model demonstrated the highest performance, identifying 13 out of 14 defects, representing a 93% success rate. The system generated 23 remarks, with 21 confirmed by engineers, resulting in a 91% accuracy rate. In contrast, the Qwen3-VL 8B model identified only one partial defect while generating 47 remarks.
The experiment highlighted that AI performance depends on both the model and the testing methodology. For instance, the GPT-5.6-sol model produced between 5 and 12 remarks in four separate runs on the same document set. Signal experts concluded that AI should be used for preliminary checks rather than replacing human engineers.
Signal currently offers an AI-based tool called Checker as a standalone application. The company plans to integrate this technology into its Signal Docs platform. The goal is to automate routine tasks, such as verifying specifications and cross-referencing data, to reduce the manual workload for engineering teams.
