Phishing triage

Triage of an email someone reported as phishing, where rules, a model and a security analyst decide together and every step is on the record: your mail pipeline makes the sender a subject with the scan, DMARC and reputation results and the message's text, events count how often the sender is reported, a model reads the message, and what neither the rules nor the model clear goes to a security analyst. Pack: examples/policies/phishing-triage/.

A pattern for expressing an email and messaging policy, not security guidance: the thresholds, questions and fields are placeholders chosen to make the example run; a real policy comes from the business's own rules. Its read step calls one model app install, chatgpt (The model step).

The decision it automates

Whether to release the message (approve), send it to an analyst, or quarantine it (reject), when your mail pipeline starts the run. Only a rule quarantines on its own: the model's doubt sends a message to an analyst, never to quarantine.

Data expected

• The sender as a subject (kind user) with email, reputation and lastMessage, written by your mail pipeline before it starts the run: dmarc and linkScanMalicious for the rules, and subjectLine, text (the body as plain text) and reporterNote (what the person who reported it wrote) for the model. Without text, the model is not asked and the rules decide alone.
• message.reported events carrying the sender's address in the promoted email key, posted through POST /events.

Steps and rules

Step	Type	Why
rules	evaluate_rules	The phishing-triage rule set under max_severity; nothing is collected.
route	branch	rules.outcome == reject (a malicious link) quarantines at once; a message with lastMessage.text goes to read; without one, manual_review opens a case and the rest is released.
read	call_app	chatgpt's decide: two questions about the message, answered under ai (The model step).
triage	branch	rules.outcome == manual_review opens a case; ai.answers.ordinary.probability >= 0.8 releases; the default (a model that sees a lure, or is unsure) opens a case.
review	create_case	security_triage, priority high, 30-minute SLA, queued by the model's lure (queueFrom: ai.answers.lure.value); case.sla.breached can page the on-call analyst.
decide	emit_decision	From the rules, or the analyst's decision.

The rule set phishing-triage (useCase: monitoring) uses max_severity: a failed block rule rejects, a failed warn rule sends the run to review, and the weights add up to the score.

Rule	Type	Severity, weight	Fails when
phish_links_clean	comparison	block, 100	lastMessage.linkScanMalicious is not false
phish_authentication	comparison	warn, 30	lastMessage.dmarc is fail
phish_sender_reputation	score threshold	warn, 30	reputation is below 40
phish_report_burst	velocity	warn, 20	more than five message.reported events for the sender's address in 24 hours

The model step

read asks the model two questions about the message. Its state is the step's variables as one JSON object, mapped from the subject: from (subject.email) and, from subject.lastMessage, subjectLine, text and reporterNote. A field the pipeline leaves out is left out.

Question	Type	Asks	Used by
ordinary	yes or no, threshold 0.8	Is it an ordinary message that asks nothing unusual of the reader?	triage: 0.8 or more releases a message every rule passed
lure	one of credentials, payment, malware, spam, none	What does the message try to get the reader to do?	review: the case's queue

The model can hold back a message the rules would release, never release one they hold: a message with a malicious link never reaches it, a message the rules send to an analyst goes to an analyst whatever the model answers, and its yes releases only a message every rule passed. triage routes on the yes-or-no's probability alone; lure only names the queue, so a choice answered without a probability changes nothing (Routing on an answer).

What the vendor sees

read sends OpenAI, through the chatgpt install, the two questions (their instructions, criteria and options) and the state: the sender's address, the subject line, the message's text and the reporter's note, and nothing else of the run. The message and the note may name people. The message is the sender's, often written to deceive: an instruction in it is text the model reads, and at worst it gets a release the rules alone would have given (Prompt injection). OpenAI's own terms say what it keeps; the console's preset calls its Responses API with storing off, decisions included (Data protection).

The chatgpt install

The pack declares the install it calls, "installs": { "chatgpt": "chatgpt" }; installs are never part of a pack. The admin console's import result installs chatgpt in one click: ChatGPT (OpenAI API) on OpenAI's Responses API, with a sample key in place of yours. Replace the key with your OpenAI API key on the install's page (Connect › Apps › ChatGPT (OpenAI API) › chatgpt, under Keys) or through POST /app-installs/chatgpt/secrets/openaiApiKey.

Until then, a run whose path reaches read fails there and no case opens; so does a run whose model call fails for another reason, such as an outage that outlasts the retries, an exhausted quota or a refusal. The vendor's answer shows in the invocation record and under the run's app calls. Once the key is in, Replay from step on the run's page runs it again from read (Admin console: Runs). A message with a malicious link, or one without text, never reaches read: the rules decide those runs whatever the key.

pnpm policy:import reports the install as missing in a workspace without one, where a run that reaches read fails with unknown_install: create it as Installing a model app shows, named chatgpt. Every model app takes the same input and answers in the same shape, so read can call an install of claude or gemini instead, declared in installs with its app, or keep chatgpt with a fallback on a second install that answers while OpenAI is down (Falling back to another install).

What a reviewer sees

A security_triage case in the queue the model's lure names, with the rule results, the model's answers under ai in the run context (each with its probability) and the sender's timeline of reports. The analyst's reject quarantines; decision.created tells your mail gateway to purge the message.

How to adapt it

Call your link scanner, reputation service and mail API from the workflow, each through an http-api install for its API (Adding a vendor step), instead of writing their answers on the subject, and add a call_app step after the decision that purges the message from every inbox. Move the 0.8 in triage once a backtest of past reports shows where your analysts and the model disagree, and create a queue for each label lure gives (Admin console: queues and filters).

Run it

pnpm policy:import examples/policies/phishing-triage --publish

pnpm policies:check runs three fixtures. The model stub (scripts/ai-stub.mjs) answers for OpenAI, following the cue word in each message's text ([[stub:low]], [[stub:high]]); the pack carries none.

• expect.json: a message that failed DMARC, from a sender with a reputation of 35, reported twice, whose text asks the reader to sign in. The authentication and reputation rules fail, the model answers ordinary at 0.05 and lure credentials, and a security_triage case opens in the credentials queue. The check quarantines it (reject) and the run completes with a manual reject.
• expect-released.json: a benefits reminder that passes every rule, which the model reads as ordinary (0.95). It is released without a case: an automated approve.
• expect-held.json: a mailbox-quota message that passes every rule, which the model holds back (ordinary 0.05, lure credentials). A case opens, and the check quarantines it.