A QA engineer utilized a multi-agent system based on CrewAI, Python, and GigaChat to audit a database of 2,000 test cases. The project aimed to identify duplicates, clarify outdated documentation, and address missing test coverage without manual rewriting.
The system employed three specialized agents: an analyst (Fedya) to identify gaps, an author (Serega) to draft test steps, and a reviewer (Sasha) to validate outputs. The workflow integrated ChromaDB and Sentence Transformers to retrieve relevant documentation chunks, which were limited to approximately 400 characters each.
The developer encountered challenges with AI hallucinations, where agents introduced unverified preconditions or misinterpreted source material. To mitigate this, the team shifted from free-form generation to a programmatic assembly of test fields using confirmed data cards.
The final validation process included strict programmatic checks to ensure each test case received a unique verdict and that duplicates were identified by content rather than titles. One specific release build successfully processed 178 test cases using this refined pipeline.
To improve reliability, the team implemented checkpointing for long-running tasks, allowing the process to resume after interruptions. The system remains dependent on human oversight, as the reviewer agent occasionally approved plausible but incorrect scenarios.
