The Unbounded Investigation

One alert, many defensible investigations. Each rung shrinks the risk. None of them drives it to zero.

A risky sign-in alert lands in the queue. One of your people, call her Dana, just authenticated successfully from a city and IP address she has never used before. Multi-factor passed. The risk engine scored it medium.

Is this Dana on a trip, or someone with Dana’s password? You investigate. And what you cannot do is finish. There is no point at which that investigation is complete. There is only the point where you decide to stop.

Watch it climb.

The fastest defensible check: has Dana ever signed in from this location before? If she has, this is probably noise. Close it. For a medium-risk sign-in with MFA satisfied, plenty of good analysts stop right here, and no reviewer would fault them.

But you could go further. What device was it? Her enrolled, managed laptop, or something nobody has seen before? An unmanaged device changes the story.

Further. What did that session actually do once it was in? Create a mailbox forwarding rule, consent to a new OAuth app, register a second MFA method, download a directory? Those are the opening moves of an account takeover, and none of them show up in the sign-in log.

Further. Has that same IP, network, or device fingerprint touched any other accounts in your tenant? One odd login is noise. The same source against forty accounts is a password-spray campaign.

Further. Just ask her. Call Dana out of band and confirm she logged in from that city at that hour.

And if you do not trust the answer? A real intruder might already be sitting in the account, or might have talked Dana into approving it. So you check with her manager, look at whether HR has her traveling, see if her role just changed.

Even that is not the bottom. Assume the account is not the prize but a doorway. Now you hunt: what can Dana reach, is there persistence, and, going back ninety days, are there earlier events that looked like nothing at the time but read as initial access now that you know to look?

You can always go one rung further. Enrich the IP against threat intelligence. Pull EDR telemetry off the device. Check her credentials against breach dumps. Talk to her team.

Two things about that ladder matter.

First, every rung is a legitimate place to stop. Hand this one alert to five good analysts and they will stop at five different rungs, each producing a clean, evidence-backed, reviewer-approved report that closes it benign. All five did good work. They just bought different amounts of certainty.

Second, each rung lowers the chance you are looking at a real intrusion, but it never lowers it to zero. There is always a more patient adversary who would have survived the check you just ran. The one who used Dana’s own managed laptop. The one who social-engineered the manager too. The one whose persistence sits one query deeper than you looked. The residual risk falls with every step and approaches zero without ever getting there.

You don’t finish an investigation. You abandon it.

So the investigation has no natural floor. You do not complete it. You abandon it, at whichever rung your time and budget give out. “Benign” is not a verdict the evidence forced on you. It is the name you give to stopping.

This is not a knock on analysts, and it does not improve when you swap the analyst for an AI agent. The shape of the problem is identical. It is also the hard end of a more general pattern: some work you can check in seconds, and some you can never check at all. Security sits at the far edge of it, which is exactly why it is worth making precise.

Investigation is abduction, and abduction doesn’t terminate

There are two ways to reason toward a conclusion, and security lives in the harder one.

Writing an exploit is deductive. You have a goal, you have the rules of the system, and you either got in or you did not. When an attacker succeeds, they know it. The proof is standing in front of them. They are inside. Verification is free.

Defending is the opposite. You are handed a pile of benign-looking logs and asked to reconstruct whether, somewhere in them, there was intent. That is abduction: inference to the best explanation for evidence you did not generate and can only partly see.

Abduction has a property that deduction does not. It never closes. There is always another hypothesis that would explain the same events, and always another event you could pull that might support it. You do not arrive at the best explanation. You arrive at the best explanation you had time to find. That is Dana’s ladder, restated as logic: it has no last rung.

One consequence is worth naming. An attacker who breaks in gets a clean signal that they succeeded. A defender who clears an alert gets nothing of the kind. Train AI on both sides and offense improves far faster than defense, because only one side has an answer key to learn from. That gap is its own subject, and I will come back to it.

Malicious has a proof. Benign doesn’t.

A threat has a finite proof: one witness. “Benign” has none. You would have to rule out every witness, forever.

Underneath the ladder is an asymmetry.

“Malicious” is provable. It has a finite witness. One renamed binary, one beacon to a command-and-control domain, one archive staged before an exfiltration, and you are done. You found the smoking gun. The claim is settled, and anyone can check it.

“Benign” is not provable. To actually establish that nothing is wrong, you would have to rule out every witness, across every event, every system, and every span of time. That is not a large search. It is an unbounded one. There is no finite piece of evidence you can hold up and say: this proves nothing happened.

So the two verdicts a SOC produces are not two sides of one coin. “Malicious” is a theorem you prove by exhibiting a witness. “Benign” is a bet you place because you ran out of places to look. This is why the residual risk on Dana’s alert shrinks but never reaches zero: there is no witness for its absence.

It is also why a security team can confirm a true positive but can never truly confirm a true negative. Catching a threat comes with a certificate. Clearing one does not. The theoretical maximum investigation, the one that would actually earn the word benign, is indistinguishable from omniscience: reconstruct the full ground truth of the environment, forever. Nobody runs that investigation. Everybody stops early. “Benign” is the name we give to stopping early.

Which means opposite investigations can both be “good”

We already watched this happen. Five analysts, five rungs, five clean reports, all closing Dana’s alert benign, at least one of them wrong. That is not sloppiness. It is under-determination: the evidence does not fix the verdict, the search path does.

Give the same alert to two competent investigators, human or AI, and they will walk different branches of the same tree and return with different, internally consistent stories. Both cite their evidence. Both are defensible. At most one matches reality, and nothing in the write-ups tells you which.

Which is why review cannot save you. Review reads the write-up. A write-up is a coherent narrative over the path that was taken. It says nothing about the paths that were not. Review grades the write-up, not the world. You cannot rank two investigations by reading them, because completeness is the one property an unbounded search can never demonstrate from the inside.

The consequence: your miss rate is invisible

Because “benign” is a bet and not a proof, a closed alert is a disposition, not an outcome. It means “someone looked and did not find anything in the time they had.” That is all it means.

So consider the most dangerous case in security: a real attack that was investigated and missed. In your data, it looks identical to a correct close. Same status, same tidy write-up, same green check. There is no field that separates “we looked and it was clean” from “we looked and we were fooled.”

This is brutal for anyone trying to measure a SOC, or train an AI on its history. Your archive of closed alerts is not a record of what was safe. It is a record of what you stopped looking at. Train on it and you do not learn to detect reality. You learn to reproduce your own misses. And you are blindest exactly where it matters most, because a competent adversary spends their effort making the worst thing look the most boring.

The miss rate, the single number that actually measures whether your defense works, is the one number your operational data cannot show you.

The only escape: plant a witness you control

If you cannot prove the negative from the inside, you have to smuggle a positive in from the outside.

You cannot verify “nothing is wrong.” But you can make something wrong, on purpose, and then measure whether anyone catches it. Plant a known-malicious artifact into the environment: a specific malicious process, a lateral move from one host to another, a beacon to a domain you own. For that one case, you are no longer guessing. You wrote the answer key.

Now “we did not find it” stops being a disposition and becomes a verified miss. The unfalsifiable “is anything wrong?” becomes the falsifiable “did you find the wrong thing I planted?” This is the only clean way to measure a false-negative rate, because it is the only way to know, for certain, that there was something to find.

Every mature engineering discipline eventually made this move. Software has the test suite: you assert a known expected output and check that the system produces it. Netflix built Chaos Monkey rather than wait for real outages to tell them whether their systems were resilient. They manufactured failures with known ground truth. Penetration testing already works this way, which is why it feels so much more honest than a compliance audit. You got in, or you did not. There is no disposition to launder.

Detection never built its version of this. The industry substituted process for outcome instead. NIST, SOC 2, MITRE coverage, “show your work” in the ticket. None of them certify that you caught the threat. They certify that you followed the procedure believed to catch threats. The verifiable thing and the valuable thing came unbonded decades ago, and everyone optimized the one they could see.

The honest coda

Two caveats keep this from becoming its own kind of faith.

The first is that even the escape only tests presence, never absence. You can grade the search for a witness you planted. You cannot grade the search for the one you never thought to plant. A planted flag measures whether you catch the attack you already imagined, not the novel one that falls outside your imagination. Planting shrinks the unknown. It does not close it, because nothing can. The asymmetry that started this essay reaches all the way to the fix.

The second is that the answer key itself is manufactured. When you decide what counts as the malicious artifact, you are encoding your own model of what an attack looks like. The answer key is an approximation of reality, not reality itself.

But an approximation you can measure beats a disposition you cannot. That is the whole move, and it is available to anyone defending anything.

Go back to Dana’s alert one last time. You will never prove that sign-in was safe. So stop trying to. Plant something unsafe instead, somewhere in the same environment, and count how often it gets caught. It is the only number in security that cannot quietly lie to you, precisely because you are the one who made it true.