Purple Team Runs

A purple team run is a coordinated exercise that evaluates defensive visibility and response against an agreed validation objective. When enabled for the organization, a plugin provides the assessment capability. Availability depends on the organization's access and the configured plugin; it is not included automatically with every AI assessment.

The purpose is to understand the relationship between observed activity, telemetry, detections, and response. An application finding and a detection gap are different results and should be reviewed separately.

Compare the assessment purposes

Aspect	AI security assessment	Purple team run through a plugin
Main question	What supported security observations should the team review?	Did the intended defensive controls observe and respond to the agreed validation?
Typical context	Project scope, application context, profile, and coverage objectives	Validation objective, environment, available telemetry, and defensive expectations
Evidence	Observations supporting candidate findings and remediation review	Execution records, telemetry, detections, and response records
Follow-up	Accept or reject candidates and track remediation	Investigate detection or response gaps and verify improvements

A purple team exercise does not replace an application assessment. An application assessment also does not establish that monitoring and response would detect the behavior evaluated in a purple team exercise.

Roles and coordination

The provider or exercise coordinator defines the authorized validation objective, confirms feature availability, and coordinates the exercise with the environment owner. The environment owner confirms the approved environment, time window, and access requirements. The defensive team identifies the relevant telemetry and the expected detection and response outcomes.

The reviewer reconciles the records from the run with the defensive team's evidence. These roles can be performed by the same people in a small team, but their responsibilities remain distinct.

Provider and client visibility can differ. A client should see only the validation options and results made available to its organization. Plugin administration and access setup remain with the authorized administrators.

Define the objective before execution

Write down the defensive question you want to answer. Examples include whether a control produces the expected telemetry, whether an alert reaches the responsible team, or whether the response process records the expected follow-up.

Document the environment and agreed success criteria so the reviewer can judge the result. A useful record includes the objective, permitted environment, expected evidence sources, relevant time window, responsible reviewers, and conditions that would make the result inconclusive.

Avoid treating every successful execution as a successful defense validation. The run may complete while the telemetry needed for evaluation is missing.

Run types and depth

Purple team describes the purpose of the exercise. It should not be confused with Automatic, Assisted, or Checklist in the AI run form. A plugin can present different controls and prerequisites from an ordinary AI run.

A single focused validation answers a narrower defensive question. A broader exercise can review multiple objectives, but each still needs its own evidence and outcome. A repeat validation checks an improvement against the earlier baseline. Scheduling, where offered, creates another execution and does not eliminate the need to coordinate review.

Use only the options actually made available by the plugin for the organization. The presence of an AI profile does not imply that the plugin supports it, and a plugin catalog entry does not imply that all execution prerequisites are ready.

Availability and readiness

The organization needs the feature enabled, an available validation option, the necessary permissions, and an environment that meets the plugin's readiness requirements. Where a local component is required, the environment owner should use the supplied setup instructions for that specific environment.

The documented AI connector and a plugin's local component are separate concepts. Do not assume that either can substitute for the other. See Local Runs for the distinction between private-network access and local execution.

If no compatible option appears, ask the provider to review access and setup. A connection problem, missing prerequisite, or unavailable telemetry should remain visible as an incomplete or inconclusive outcome rather than being recorded as a passed control.

Monitor the run and collect the evidence

Keep the run's execution status separate from the defensive outcome. Review whether the intended validation occurred, whether the environment was ready throughout the relevant period, and whether the required evidence is available.

Where the plugin exposes execution history or artifacts, use them with the defensive team's telemetry and incident records. Correlate the relevant environment and time window. An execution record alone cannot establish whether an alert was generated, delivered, investigated, or resolved.

Interpret outcomes separately

Result category	Question for the reviewer
Execution	Did the agreed validation occur, or was it blocked or incomplete?
Telemetry	Was the expected observation recorded and available for review?
Detection	Did the intended control produce the expected detection?
Response	Was the expected investigation or response carried out and recorded?

These categories can have different outcomes in the same run. Activity can occur without the expected detection. A detection can occur without a completed response. Missing telemetry can make a conclusion impossible even when the execution completed.

Use clear outcomes such as supported, not supported, blocked, and inconclusive in the exercise report. Preserve the evidence and explain why the reviewer reached the conclusion. Do not claim a broader level of protection than the exercise established.

Findings and remediation

A defensive gap should identify the reviewed control, expected outcome, actual observation, evidence, and the team responsible for follow-up. It may call for a configuration change, telemetry improvement, alert review, or process correction.

Track accepted work through the organization's issue process when that workflow is available. Keep issue status separate from run status. A completed purple team run does not mean that its follow-up work is closed.

After improvements, compare a repeat validation with the original objective and evidence. Record changes in the environment or validation conditions so that the comparison remains meaningful.

For AI interaction modes and result review, see AI Runs and Run Modes.