judgment as owned infrastructure
Tom checks the calls the AI in your SOC has already made against what actually happened, and shows you where the evidence supports letting it act alone.*
Beside your SOC today. Plugged into your stack tomorrow.
* Tom reads decision records only, read-only, inside your cloud. It judges agents, not analysts, and nothing changes in your SOC without your sign-off.
CyberSOC · for CISOs at regulated firms
Tacit Labs is the independent judge for the AI in your security operations centre. We check its calls on your own alerts against what actually happened, and show you where the evidence supports letting it act alone.
| Alert type | Call | Why |
|---|---|---|
| User-reported phishing | Act alone | Right on outcomes. Decides like your team. |
| Unfamiliar sign-in | Act, and log for review | Mostly right. Some calls need a later look. |
| Impossible travel | Recommend; a person decides | Misses context your team uses. |
| Privileged role change | Off | Breaks a rule your team always follows. |
AI triage now comes inside tools and services you already pay for. Since November 2025, Microsoft has been adding Security Copilot to Microsoft 365 E5 licences.1 Managed detection and response (MDR) providers have built AI into their services.
Once it’s set up, an AI triage agent can close the alerts it judges harmless. Microsoft’s phishing triage agent, for example, resolves the emails it judges non-malicious, so analysts don’t spend time reviewing them.2
It doesn’t arrive tested on your alerts. And if it’s wrong, the regulator’s questions come to you.
From a 2026 survey of security leaders.3
19%
rely on the vendor’s own accuracy figures to check AI verdicts.3
57%
still require a person to review every AI verdict before an alert is closed.3
46% vs 17%
In a Microsoft trial with planted errors, analysts who saw the AI wrongly clear a malicious email missed it 46% of the time. Without the AI, they missed 17%.2
Only one in five leans on the vendor’s number. Re-checking everything is prudent, but it eats the time the AI was meant to save. And re-checking with the AI’s verdict on screen catches far fewer of its misses.
To be fair to the AI: the same Microsoft trial found it made analysts far more accurate when it was right.2 The problem isn’t the AI. It’s not knowing when it’s wrong.
That’s Tacit Labs. A house judge works for you. It doesn’t build the AI, sell it or run it.
It watches the AI’s calls on your alerts and tells you, with evidence, how far to trust it. Think of it as the interview your AI analyst never had, and the check-ups it never gets.
The method is Bayesian. In plain words: it starts with a best guess, updates with every decision and every outcome, and always knows how sure it is. When the evidence is thin, your report says so. We never round up.
The proof most often offered for an AI analyst is how often it agrees with a human one. Here are two published figures.
99.7%
How often an AI agent agreed with human analysts in one vendor’s customer trials, across 138,475 alerts.4
67%
How often analysts agreed with an AI agent’s verdict in a 14-month study at a university SOC: 72 of 108 tickets. Of the seven outright clashes, only one was a clear AI error. The other six were judgment calls.5
Both can be true. Different AIs, different alerts and, above all, different people to agree with.
Analysts disagree with each other, and anyone can call the same alert differently on a different day. So “agreement with an analyst” is a ruler that moves.
We fix the ruler first. Where outcomes exist, we use them. Where they don’t, we use your team’s steady judgment.
That’s how your team decides on a good day. We call it the house view. Then we measure the AI against that.
It matters for learning too. Some AI agents learn from analysts’ feedback, including overrides. An override made in a rush teaches the wrong lesson.
For each alert type, your report answers six questions. Every answer says what it rests on: confirmed outcomes, a blind second look, or your team’s judgment.
We check the AI’s calls against what actually happened: confirmed incidents, re-opened alerts, red-team and attack-simulation results.
The alerts it closes alone get a blind second look. Each week your analysts re-check a small random sample in a view that hides the AI’s verdict. Seeing the verdict changes what people see.
You get: accuracy and missed-threat rate per alert type, each with a range that shows how sure we are.
Same alert, same facts, same answer? We check whether the AI decides the same way when the same situation comes back, across days and model updates.
You get: a consistency rate per alert type, and the cases where it flipped.
Vendors update models. Attackers change tactics. Your alert mix shifts.
We keep a frozen baseline. When something moves, we tell you what moved: the AI, your team or the alerts.
You get: drift alarms that name the cause. An alarm calls a person to look. It never changes anything on its own.
Every override has a reason. We sort each one into one of six causes:
You get: a teach list. It shows which overrides the AI should learn from, which it shouldn’t, and what context to connect.
We describe the AI the way you’d describe a new analyst. Where it’s stricter or softer than your team. Which signals it weighs differently. Which shortcuts it takes that your team never would.
You get: plain-language differences, each with an example and a line on what it doesn’t mean. Different isn’t wrong until outcomes say so.
For each alert type, one of four calls:
Each call shows the evidence behind it, and what would change it.
You get: an autonomy map you sign off, reviewed monthly. The four calls follow the four outcomes in MAS’s SAFR paper: Auto-Execute, Observe, Escalate and Deny.6
You apply the map in your platform’s own settings. Where a platform can’t hold its AI to “recommend”, your report says so, and the call becomes “off” for that alert type. Nothing changes in your SOC without your sign-off.
Regulated firms run on reporting clocks:
If the AI closes a real incident as harmless, nobody knows the clock should be running.
So for regulated teams we add one more number: reporting-clock exposure. For each alert type, it shows how often the AI clears alerts your team would have escalated as a possible reportable incident.
You get: your exposure by alert type, measured against your regulator’s clock.
Read-only. In your cloud. First answers in six weeks when there’s enough history.
We connect to where decisions are recorded: your XDR, your SIEM and your ticketing. Never to the AI itself. We can’t change an alert, a rule or a setting.
We read up to 90 days of decision records, matched by alert ID: the alert, the AI’s verdict, your team’s decision, the action and the outcome. Not enough history yet? We record forward for six weeks instead.
Each week your analysts re-check a small random sample of the alerts the AI closed, in a separate Tacit view that hides the AI’s verdict. Nothing is written to your systems.
We estimate about an hour of analyst time per alert type per week.
In six weeks: the six answers for each alert type, your autonomy map, and an evidence pack for risk and audit. After that, Shadow Judge can keep watching every week.
Where to start: We suggest starting with user-reported phishing. It’s high-volume, and outcomes come back fast. Identity alerts come next.
Ten moments when you need to know how far to trust the AI. Find yours.
Decide
You’re asking Which alert types are safe to hand over?
You get An autonomy map per alert type, with the evidence for each call and what would change it.
You’re asking How will we know if it helped?
You get A baseline of how your team decides today, so you can judge any AI from day one.
You’re asking Is our provider’s AI good enough to renew?
You get If the AI runs in your tenant, a report on it. If it runs on the provider’s platform, the clause to ask for a per-alert decision export.
Prove
You’re asking How do we know the AI works?
You get An independent evidence pack for risk and audit teams: accuracy against outcomes, consistency, drift and override reasons, per alert type.
You’re asking Could the AI be hiding a reportable incident?
You get Reporting-clock exposure per alert type, against your regulator’s clock.
You’re asking How do we prove our AI to each client?
You get An independent report for each client, delivered to the client unedited.
Run
You’re asking Where can we stop?
You get The alert types where the evidence supports spot-checks instead of full review, and the ones where it doesn’t.
You’re asking Did anything change?
You get A drift check against your frozen baseline that says what moved: the AI, your team or the alerts.
You’re asking Who’s right, and what should the AI learn?
You get The reasons, sorted into six causes, and a teach list for the AI.
You’re asking A one-off, or a pattern?
You get How often the AI makes that call, on which alert types, and how your team would have called it.
See which of these fit your SOC.
Book a 30-minute callYou’re accountable for AI you didn’t choose. Get evidence you can sign off on.
Six questions →Know where to trust the AI, and where to keep your people.
When you’ll want the evidence →Independent, outcome-based testing of vendor AI, with monitoring records you can file.
What your regulator is starting to ask →Show each client how your AI performs, in a report they receive unedited.
Provider licence →To judge the AI, we need its decision records. Who holds them depends on your set-up.
| Your set-up | Who holds the AI’s decision records | How we connect |
|---|---|---|
| In-house SOC on Microsoft Defender and Sentinel, or Google Security Operations Google connector: early 2027 |
You, in your own tenant | You grant read-only access. No other vendor needs to say yes. |
| A provider co-manages your SOC inside your tenant | You | The same as in-house. |
| An MDR provider works in your CrowdStrike console | Usually you | Usually direct, once our CrowdStrike connector is ready (2027). We confirm what’s recorded in one short call. |
| An MDR provider runs its AI on its own platform | Your provider | Through your provider. Ask for a per-alert decision export (we’ll give you the clause), or ask them to commission a Tacit report. Either way, the report comes to you unedited. |
Not sure which one you are? That’s the first thing we work out on the call.
Tacit is built to read from the systems where decisions land, and to judge any AI agent whose verdicts are recorded: a platform’s agent, your provider’s AI, a tool you bought, or one you built.
| System | What we read | Status |
|---|---|---|
| Microsoft Defender XDR and Microsoft Sentinel | Alerts, incidents, the AI’s classifications, audit history | First Design partners from December 2026 |
| Google Security Operations | Investigations, the AI’s verdict and confidence | Next Early 2027 |
| CrowdStrike Falcon | Alerts and decisions | Planned 2027 |
| Splunk Enterprise Security | Notable events and how they were closed | Planned |
| ServiceNow, Jira | Outcomes and case history | First |
| Your provider’s export (CSV or API) | Per-alert decision records | With the first design partners |
Running something else? Tell us what you use.
Leaving: Revoke our access and we stop reading.
An AI shouldn’t be graded only by the company that sells it.
Today, the hard deadlines are for reporting incidents. On AI, most rules are new, in draft, or start later. But they all point the same way: you stay accountable for the AI you use, including AI from a vendor.
Where Tacit helps: outcome-based evidence your own validation team can use and challenge; drift monitoring; reporting-clock exposure.
Where Tacit helps: measured evidence on third-party AI; an autonomy map that uses SAFR’s four outcomes; reporting-clock exposure.
Where Tacit helps: oversight records for the AI in your detection and response; monitoring of third-party AI.
Where Tacit helps: outcome analysis and ongoing monitoring of AI triage models.
Tacit gives you evidence for these questions. It doesn’t make you compliant with any of them. This page is not legal advice. Updated .
A judge has to be tested too. Before we judge your AI, we test our own engine on data where we planted the answers.
Our engine was first built for insurance underwriting desks. In a synthetic underwriting test (8 underwriters, 3,200 files), it found all 7 patterns we had planted, with 1 false alarm in 113 places where there was nothing to find.15
Our SOC test comes next: synthetic data from 10 analysts and about 30,000 alerts, with a sealed answer key. We’ll publish what it finds, including what it misses.
Agent Decision Audit · sample report (illustrative, synthetic data)
| Alert type | Accuracy | Missed threats | Consistency | Drift | Call |
|---|---|---|---|---|---|
| User-reported phishing Rests on outcomes | 97% 96–98 | 0.3% 0.1–0.6 | 98% 97–99 | Stable | Act alone |
| Unfamiliar sign-in Rests on outcomes and a blind second look | 92% 90–94 | 1.2% 0.8–1.8 | 95% 93–97 | Stable | Act, and log for review |
| Impossible travel Rests on a blind second look and team view | 86% 82–89 | 2.6% 1.7–3.8 | 90% 86–93 | The AI moved · week 4 | Recommend; a person decides |
| Privileged role change thin · 46 decisions Rests on team view | 81% 69–90 | 4.3% 1.4–11 | 84% 72–93 | Too few to tell | Off |
Every rate comes with a range. Thin evidence is labelled as thin.
Who’s behind it: Tacit Labs started inside Birchlogic, a CERT-In-empanelled cyber and compliance consultancy. Co-founder Karan Bhandari is a vCISO who has set up a SOC end to end.
We’re starting with a small group of design partners from December 2026, on Microsoft Defender and Sentinel.
Six weeks, fixed fee, three to five alert types. A look back at up to 90 days, plus a blind second look. You get the six answers, your autonomy map and an evidence pack.
Live and read-only: every AI verdict is scored. A weekly drift and override watch. A monthly autonomy review.
An optional mode, off unless you switch it on: the AI checks with the judge before it acts, and every call is logged. Everything else on this page stays read-only.
For MDR and MSSP providers. An independent report for each client, delivered unedited.
No AI switched on yet? Start with a baseline of how your team decides, so you can judge any AI from day one.
Design partners: We’re taking on three design partners.
No. Tacit sits beside them and reads where decisions are recorded. It never sits between your analysts and their tools.
No. We judge agents, not analysts. Analyst IDs are scrambled with a key only you hold, and every result about people is reported at team level.
Yes, through the provider. You have two options, and either way you get the report unedited:
We read decision records, and only the fields you approve. We list them before we connect:
User names, email addresses and IP addresses are hashed with your key. We don’t read email bodies, attachments, files or packet captures. Nothing leaves your cloud except the reports you choose to share.
Up to 90 days. For a firm answer on an alert type, we need a few hundred AI decisions on it. If you have fewer, we record forward for six weeks.
It’s designed for exactly that, because it’s normal in a SOC. We use the outcomes that do exist, the blind second look and your team’s steady judgment. Each finding says which it rests on, and thin evidence is labelled as thin.
A vendor’s dashboard reports on its own AI, by its own measure. Tacit measures it independently, on your alerts, against what actually happened. It also shows why your people overrule it and where it can act alone.
Those test whether planted attacks get caught. They’re useful, and we use their results as outcomes. But they can’t tell you why your people overrule the AI, whether it’s drifting on real traffic, or where the evidence supports letting it act alone.
No. Those tools use another AI to grade an AI. Tacit checks your AI’s calls against what actually happened on your alerts, and against how your own team decides.
It starts with a best guess, updates with every decision and every outcome, and always knows how sure it is. So every number in your report comes with a range, and thin evidence is labelled as thin.
No tool can. Tacit gives you evidence your frameworks ask for: independent testing, monitoring, and records of human oversight. Your compliance team decides how to use it.
Yes. Under NDA, we share our method documentation, test results and the evidence behind every finding. Your validators can review it like any other third-party model.
We’ll complete your security questionnaire and share a data-flow diagram before we connect.
The Audit is a fixed fee. Shadow Judge is a subscription priced per agent. Provider licences are priced per client. Talk to us for a quote.
Tacit Labs started inside Birchlogic, a CERT-In-empanelled cyber and compliance consultancy working across India, Singapore, Canada, Germany, Ireland and the Netherlands. Co-founder Karan Bhandari is a vCISO who has set up a SOC end to end.
Book 30 minutes. Tell us your platform, your provider and which AI is switched on. On the call, we’ll tell you whether we can connect, what we’d measure first and what you’d get.
Pick a time that suits you. No deck, no obligation.
Prefer to write? Email hi@tacitlabs.ai.
