A tutor is developing AI assistants for two projects: MathExam and a literature study group. The goal is to ensure the AI provides guidance without solving problems for students or spoiling literary plots. The developer is testing models from OpenAI and Kimi to create consistent instructional prompts.

The testing process involved creating specific instructions to prevent the AI from providing direct answers. For example, in a division problem, the model was instructed to guide the student through steps rather than stating the final digit. Similar adjustments were made to the literature assistant to prevent it from summarizing endings or imposing interpretations.

The developer built a Python-based runner to manage API requests and evaluate model responses. The testing framework evolved through several versions, eventually incorporating 57 automated tests and 13 agent-based checks. The project maintains a corpus of conditions, responses, and revision histories.

As of November 20, the project recorded 2,264 API responses with an estimated cost of $45.91. The developer noted that while the budget is being used efficiently, technical issues such as HTTP errors have occasionally interrupted the automated testing process, requiring further analysis before scaling.