A developer and ЕГЭ expert created a Telegram bot to grade the second part of the social studies exam. The system uses an AI model for text analysis and labeling, while a hard-coded script handles the final scoring. This architecture ensures that the model cannot unilaterally assign a grade, as the code can only reduce the score based on formal criteria.
The bot uses specific plugins for each task, ranging from simple keyword checks to complex plan analysis. For instance, in task 24, the system lemmatizes plan points using pymorphy2 and compares them against a database of 185 standard plans. To prevent false negatives, the system employs synonym groups for headings, ensuring that variations in terminology do not result in a zero score.
To maintain reliability, the system implements strict filters for error detection. Findings from the model must pass multiple checks, including consensus between independent runs and verification against a list of known error types. If the model fails to return a valid JSON or encounters a network error, the system flags the check as unavailable rather than assigning an incorrect score.
In a November test run involving 393 cases, the system performed 497 model calls at a total cost of $8.87. On average, each check required 1.26 calls, cost $0.023, and took 9 seconds. The developer noted that re-checks are the most expensive part of the process, and prompt caching significantly impacts costs depending on the length of the system prompts.
