Igor Zuriev, an expert in AI-based business automation and IT project management, outlines a framework for selecting and evaluating Large Language Models for corporate tasks. The process begins with reviewing public benchmarks, followed by testing candidate models against specific internal requirements and defined acceptance criteria.
The evaluation methodology emphasizes practical utility over theoretical performance. Key metrics include the accuracy of the output, the necessity for manual corrections, and the overall efficiency of the model in handling complex workflows, such as multi-file code analysis or document processing with evolving business logic.
Zuriev shares personal experience with various tools, noting that Fable 5.1 demonstrated high accuracy in code generation. He also highlights the use of Stitch for interface prototyping and mentions encountering operational constraints, such as hitting usage limits with Sonnet 5 during complex tasks.
