AI agents in bug hunting often generate false positives, including fabricated vulnerabilities and inflated severity ratings. To mitigate this, researchers must provide agents with direct access to tools for validation, such as isolated virtual machines or MCP containers, rather than relying on speculative output.
Effective validation requires manual verification of findings using tools like Burp or curl. Implementing strict reporting templates forces agents to provide concrete evidence, while custom rules for specific vulnerability classes help define clear criteria for what constitutes a valid bug.
To improve accuracy, researchers should provide agents with detailed application context and employ a two-agent system where the second agent attempts to refute the findings of the first. This approach helps filter out speculative chains and ensures that reported vulnerabilities reflect real business impact.
Managing costs in agentic bug hunting involves using a tiered model architecture: expensive, high-capability models handle complex analysis and planning, while cheaper models perform routine scanning. Maintaining a database of past findings and failures further optimizes token usage and prevents repetitive errors.
Scope management is critical to avoid legal and professional repercussions. Researchers should define clear operational modes for agents—such as wildcard, domain list, or solo-domain—to prevent them from scanning unauthorized targets and to ensure compliance with program rules.
