The cost of generating AI-related content has dropped significantly, as models can produce expert-sounding text for mere cents. However, verifying these claims remains expensive and time-consuming, as there is no independent anchor or test to validate predictions, unlike software code which can be verified by compilers or logs.
Several studies highlight the discrepancy between AI-assisted performance and perceived productivity. Research from METR indicated that developers using AI tools completed tasks about 20% slower than those without, despite believing they were faster. The organization later noted that evidence for AI-driven acceleration remains weak and requires further testing.
Other reports, such as those from GitClear and Veracode, suggest potential issues with code quality and security when using AI. GitClear observed an increase in code copied without modifications, while Veracode reported that models often chose insecure options over secure ones. These findings are often used to drive engagement through fear-based narratives.
Real-world implementation of AI systems requires significant manual effort beyond simple code generation. For instance, building an AI-based real estate parser took several days, while an AI complex for a studio required three weeks of development. Business logic, system integration, and error handling remain manual tasks that cannot be fully automated.
System maintenance and verification are critical, as AI models can hallucinate non-existent libraries or suggest incorrect configurations. Automated reports generated from telemetry often contain internal inconsistencies, such as conflicting response times or mismatched dialogue counts, requiring hours of manual review to ensure accuracy.
