Docs / Monitoring / Incident Response with AI Pilot

Incident Response with AI Pilot

Published · Last updated: May 2026 · 4 min read

Who this is for

Anyone who has received a critical alert and wants to use AI Pilot to diagnose and respond to the incident faster.

What you will complete

Use AI Pilot as your first responder for a firing alert — from initial diagnosis through to proposing and executing a fix.

Before you begin

  • AI Pilot must be enabled with Read Access (at minimum) for diagnosis.
  • Write Access required to execute a fix via AI Pilot.
  • Navigate to AI Pilot in the left sidebar.

AI Pilot as a first responder

When a critical alert fires, AI Pilot can:

  1. Read the alert context — which server, which metric, when it started
  2. Pull the relevant logs and current metrics
  3. Diagnose the most likely cause
  4. Propose a remediation action for your approval

This does not replace human judgment — you always approve or deny. But it compresses the investigation from 15–30 minutes of manual log diving to 2–3 minutes of AI-assisted diagnosis.

Step-by-step: incident response workflow

Step 1: Acknowledge the alert

  1. Go to Alerts → Events tab.
  2. Find the firing alert.
  3. Click Acknowledge to signal you are investigating.
  4. Optional: click Snooze for 1 hour to suppress repeat notifications while you investigate.

Step 2: Open AI Pilot

  1. Go to AI Pilot in the left sidebar.
  2. Describe the incident with the server name and metric. Example:

"I have a critical CPU alert firing on server-name. It has been above 95% for the past 10 minutes. Please diagnose what is causing it."

Step 3: Review the AI diagnosis

AI Pilot will:

  • Read the current CPU metrics and recent history
  • Check active processes and load average if SSH access is enabled
  • Review recent deployments and operations that may correlate
  • Identify the most likely cause (runaway process, deployment spike, resource exhaustion)
  • Explain the diagnosis in plain English

Step 4: Approve the proposed fix (if applicable)

If the AI identifies a fixable cause, it will propose an action. Common proposals for CPU incidents:

  • Restart the specific service consuming excess CPU
  • Kill a specific runaway process
  • Trigger a deployment rollback if the spike started with a recent deployment

Review the approval card carefully. If the proposed fix looks correct, click Allow.

Step 5: Verify resolution

After the fix executes:

  1. Watch the CPU metric in the Monitoring dashboard — it should begin dropping.
  2. Within 1–2 minutes, if the fix was effective, the alert should auto-resolve.
  3. Ask AI Pilot: "Can you confirm server-name CPU is back to normal?"

Step 6: Post-incident note

Ask AI Pilot: "What happened on server-name during this incident and how can we prevent it?" The AI will summarize the incident and suggest a monitoring rule, goal trigger, or configuration change to prevent recurrence.

What success looks like

  • Alert transitions from Firing to Resolved within minutes of the fix executing.
  • CPU (or other affected metric) returns to normal range in the monitoring chart.
  • The Activity Center shows the AI-initiated action as Completed.

Common errors and fixes

"AI Pilot says it cannot read logs for this server" Cause: File and Log Access is disabled, or the server's per-server AI access is set to Read Only or No Access. Fix: Enable File and Log Access under Settings → AI Agent → Agent Controls, and verify the server's per-server access level.

"AI Pilot diagnosed the issue but cannot propose a fix" Cause: The required write operation category may be disabled (e.g., Services and SSH operations are off). Fix: Check Settings → AI Agent → Agent Controls → Fine-tune Write Actions → Server Operations → Services and SSH.

"The alert did not resolve after the AI executed the fix" Cause: The AI fixed one cause but the metric is still elevated due to a secondary issue. Fix: Ask AI Pilot: "CPU on server-name is still high after restarting nginx. What else could be causing it?"

Related articles