anthropics/oncall-kit

anthropics/oncall-kit

Description: Starter kit for a Claude-assisted on-call: mines your incident history into triage playbooks, sets up through human-approved gates, and runs read-only in your Slack channel — humans deploy every fix.

Language: Python

License: Apache-2.0

Stars: 5

Forks: 0

Open issues: 0

Created: 2026-08-03T22:18:23Z

Pushed: 2026-08-06T19:11:35Z

Default branch: main

Fork: no

Archived: no

README:

On-Call Kit


Put Claude in your team's incident channel. When an alert fires, it investigates and posts a first-pass diagnosis with a link behind every claim. A human decides what to do. Claude then watches the fix land and writes down what was learned, so the next incident starts smarter.

> Reference implementation. Not maintained and not accepting contributions.

This is the kit behind the blog post on building a CI on-call with Claude: the playbooks, templates, and a guided setup that drafts those playbooks from your team's own incident history. (Throughout: a skill is a markdown instruction file that tells Claude how to do one job — triage an alert, write a handoff. You review and change skills like code.)

Two properties worth knowing up front:

• It works with whatever tools you already use. No skill names a

specific vendor. Skills refer to *capabilities* — the metrics source, the log store, the pager, the code host — and one generated file, STACK.md, maps each capability to the tool your channel actually has connected. Swap Datadog for Grafana, or PagerDuty for Opsgenie, and no skill changes.

• You don't write the playbooks from scratch. Setup reads your

team's past incidents — alert threads, pager history, postmortems — and drafts the playbooks from what your team actually did. You review and correct the drafts.

Try it in 10 minutes

Zero connections, zero admin. Clone this repo anywhere, open Claude Code in it, and paste the one instruction from test-fixtures/RUNBOOK.md. You'll watch the whole setup run against a fictional team's 48-incident history: mining, playbook drafting, the sign-off gates, a graded validation against an answer key. Nothing real is touched, so there is nothing to clean up if you stop here.

The fixture doubles as the kit's regression test: change a skill, re-run it, diff against the key.

What you get

One Slack channel where:

• an alert landing in a watched channel (or a human declaring an

incident) starts an investigation automatically

• Claude posts a grounded first-pass diagnosis — every claim linked to

the log line or dashboard panel behind it — and proposes a fix

• a human decides, a human (or gated automation) deploys, and Claude

watches the metrics land back at baseline

• anyone can ask for status on demand ("safe to merge?") and get an

answer grounded in the open incident records

• every incident appends to lessons.md, so the next investigation

starts smarter

• Monday morning, the handoff writes itself

The division of labor is fixed and non-negotiable (see CLAUDE.md): Claude gathers evidence, proposes, verifies, and communicates. Humans decide what to mitigate and when.

Words this kit uses

Eight terms, defined once:

• Capability binding — the line in STACK.md mapping an abstract

capability ("metrics") to your real tool ("Grafana").

• Routine — a standing task Claude runs in the Slack channel on a

schedule or trigger ("watch #ci-alerts", "post the handoff Monday 9am"). You create one by pasting or saying the request in the channel; anyone in the channel can list or disable them.

• Canvas — Slack's built-in document attached to a channel. The kit

uses one to list the channel's routines and, optionally, to hold the weather report.

• Shadow period — the trial phase: Claude posts diagnoses to a

review channel while humans handle incidents as before. It goes live only after its answers prove reliable.

• Holdout — a past incident deliberately kept out of playbook

drafting, so it can serve as an unseen test in the Validate phase.

• Provenance tag — the annotation on every mined playbook claim

saying how many incidents support it, e.g. (seen 2×, unverified).

• Page-tier vs. morning log — the two urgency tiers: a *page* wakes

a human now; a *morning log* line waits for the daily summary. ONCALL.md decides which is which.

• MCP — the connector protocol Claude uses to reach tools like your

metrics source or pager. Your Claude org Owner attaches these connections (see TAG-SETUP.md).

How it works: the incident loop

sequenceDiagram
participant A as Alert channel
participant C as Claude
participant H as Human on-call

A->>C: Alert fires
Note over A,H: Page-worthy alerts page the human directly - Claude is never the detector
C->>C: Classify the symptom,
load the matching playbook,
check past lessons
C->>H: First-pass diagnosis with evidence links
and a proposed fix
H->>C: Pushback - could it be something else?
C->>H: Checks the new hypothesis against the data,
reports back either way
H->>H: Decides on the fix and deploys it
C->>H: Watches the metrics return to normal,
confirms in the thread
H->>H: Closes the incident - humans always close
C->>C: Appends what was learned to lessons.md

Stage by stage:

| Lifecycle stage | Who/what | |---|---| | Detection | Your existing alerting stays primary: alerts fire into channels, and page-worthy ones page humans directly. Claude is never the detector. It watches the alert channels, triages what fires, and correlates across them — five alerts are often one incident. For a new service with no alerts yet, it drafts conservative starter rules for a human to install before launch | | Diagnosis | The triage skill: classify the symptom, load the matching reference file, check lessons.md for known causes, post a grounded diagnosis with evidence links | | Mitigation | Human decides. Claude proposes — including, where your team uses feature flags, a ready-to-execute canary ramp plan: the flag, the percentage steps, the hold time at each step, and the single metric that aborts the ramp. The human cross-examines, chooses, and deploys (or approves a gated action). The kit never touches a flag; it writes the plan a human runs | | Verification | "Watch it land": Claude polls the affected metrics until baseline, posts confirmation |...