HiveTrace Red is a tool designed for red-teaming AI applications, which involves testing systems for vulnerabilities such as unauthorized data access or restricted action execution. The platform uses a metric called Attack Success Rate (ASR) to measure the frequency of successful security breaches during testing.
To address the limitations of binary ASR metrics, the developers incorporate uncertainty assessment using the Clopper-Pearson one-sided 95% upper bound. This statistical method helps determine if the sample size is sufficient to confirm that an application meets specific security thresholds, even when no violations are found.
For example, testing an assistant with 10 queries and zero violations results in an upper bound of approximately 25.89%. If the required security threshold is 0.3%, this sample size is insufficient. To confirm a 0.3% threshold with zero violations, a minimum of 998 independent queries is required.
The methodology requires defining risk categories, security criteria, and acceptable thresholds before testing begins. If a single violation occurs, the upper bound calculation changes, which may lead to blocking an application's release if the threshold is exceeded.
HiveTrace Red supports over 90 attack techniques, including role-playing scenarios, prompt format manipulation, and multi-step interactions. These are applied to the same dataset used in baseline testing to ensure consistent comparison of results across different security configurations.
