Blog

DIY guardrails vs a runtime authorization layer: an honest comparison

· 7 min read

Vulnify blog cover: DIY guardrails vs a runtime authorization layer: an honest comparison

Most teams that ship an AI agent with real tools end up writing the same few lines sooner or later:

def export_tickets(email, rows):
    if rows > 500 or not email.endswith("@ourcompany.com"):
        raise PermissionError("export not allowed")
    ...

That's a guardrail, and it's a good one. It sits in front of the action, not the prompt, it's deterministic, and it costs nothing to run. This post is not here to talk you out of it. It's a look at what that check gets right, what it quietly turns into as the agent grows, and how to tell which side of the line you're on.

What the homegrown check gets right

Credit where it's due. A plain if in front of a tool call has real strengths:

  • It's at the right layer. It looks at the action the agent is about to take (what, how much, where to), not at what the model said. That's the layer where damage actually happens.
  • It's fast and local. No network hop, no new dependency, no new vendor to review.
  • You understand all of it. It's your code, in your repo, covered by your tests.
  • It fits exactly. It knows your domain: which tables matter, which domains are internal, what a normal export looks like.

If your agent has two or three risky tools, one team owns it, and nobody outside that team needs to see what it did, a check like this may be all you need for a long time.

What it turns into

The trouble isn't the first check. It's what gets asked of it next. Each of these is a reasonable request, and each one is new code:

"Can a person approve the borderline ones?" Now you need a review state, a queue, a way to notify someone, a rule that the person approving isn't the one who asked, an expiry so an unanswered request doesn't sit forever, and a decision about what the agent does while it waits. That's a small product, not an if.

"Show me everything the agent did last month." The check raised or didn't. Unless you also logged every decision with its inputs and reasons, there's nothing to show. And once you do log it, someone will ask how they know the log wasn't edited, and how long it's kept.

"Why was this blocked?" A PermissionError doesn't explain itself. Useful answers need the reasons attached to the decision: external destination, large volume, sensitive resource.

"Add the same rule to the other three agents." The rule lives inside one agent's code. Copying it means three versions that drift. Changing a threshold means a deploy of every agent that has it.

"Will this change break anything?" Tightening a threshold is a guess unless you can run the new rule against real past traffic or a set of test cases before it goes live.

"Is customer data ending up in the logs?" If your decision log stores the payload, your guardrail just became another copy of the data it was supposed to protect.

None of this is exotic. It's the normal path from "agent in a demo" to "agent a security review signed off on". The question is whether you want to build and maintain that path yourself.

Side by side

Homegrown check Runtime authorization layer
Where the rule lives In each agent's code Outside the agent, shared by every agent
Change a threshold Code change and deploy Edit the policy; the agent doesn't change
Outcomes Usually run or raise Allow, send to a person, or block
Explanation Whatever you write in the error Risk score and reasons on every decision
Human approval Build the queue, notifications, expiry and approver rules Built in
Audit trail Whatever you log Every decision recorded, with a way to check it wasn't altered
Testing a rule change Your unit tests Test cases against draft policies, plus a monitor mode on live traffic
Latency and dependencies None added A network call per checked action
Fit to your domain Exact As good as the actions and resources you describe

The last two rows are where DIY wins, and they're real. An external check adds a network call and a dependency you have to plan for, including what your agent does if that check can't be reached.

What the layer looks like in Vulnify

Vulnify is one way to get the right-hand column. Before a sensitive action runs, your code describes it (agent, action, resource, destination, record count) and Vulnify answers ALLOW, REVIEW or BLOCK, with a riskScore from 0 to 100, a riskLevel and the reasons. The action runs only when finalDecision is ALLOW.

The if from the top of this post becomes a call that carries the same facts:

vulnify.guard(
    lambda: export_tickets(email, rows),
    agent="SupportBot",
    action="EXPORT_DATA",
    resource="Support Tickets",
    destination="EXTERNAL_EMAIL",
    records_affected=rows,
)

On REVIEW or BLOCK, guard() raises VulnifyBlockedError and the export doesn't run. The rules move out of the agent into policies you edit in the app or keep as YAML in git, and the same policy applies to every agent that sends that action.

Here's how each item from the list above is handled, as documented:

  • Approvals. A REVIEW waits for someone with a role the policy allows, other than the requester. Approvers answer in Vulnify or in Slack, and owners and admins can get review emails. HIGH and CRITICAL approvals ask for an authenticator code when the approver has MFA on. A denied or expired review becomes BLOCK. More on keeping that queue small in human-in-the-loop without approval fatigue.
  • Audit. Every decision, including human reviews, goes into an append-only audit log on a per-organization SHA-256 hash chain, and you can run a verification that reports the first entry that doesn't match. Owners and admins can download recent events as JSON or CSV with a SHA-256 checksum.
  • No payload copies. Records hold action metadata (agent, action, resource, destination, counts, risk results, review decisions), not the customer records. Optional content sent for scanning is not stored.
  • Retention. Decision events are deleted after your plan's retention window, which is published per plan.
  • Testing changes. The vulnify CLI validates policy files offline and runs test cases with vulnify test --local, which evaluates the policies in your working copy without writing any security event. In the app you can replay recent history against a draft rule, and monitor mode records what would have been reviewed or blocked while still answering ALLOW.

When DIY is the right call

Build it yourself when most of these are true:

  • The agent has a handful of risky tools and that list isn't growing.
  • One team writes, runs and is accountable for it.
  • Nobody outside that team needs to approve actions or read a history of them.
  • You can't accept a network call in that code path.
  • Your rules are simple enough that a unit test covers them.

Reach for a runtime authorization layer when a person needs to approve some actions, when an auditor or a customer's security review will ask what the agent did, when several agents need the same rules, or when the people setting the thresholds aren't the people deploying the agent.

There's also a middle path that a lot of teams take: keep your local checks as a first line, and add the external decision only on the few actions that move data out, delete things or touch money. The two don't conflict.

A quick self-check

  • Can you list every action your agents can take that would hurt if it went wrong?
  • For each one, who decides if it runs, and where is that rule written down?
  • If a borderline one happens at 2 a.m., who sees it, and what happens if nobody answers?
  • Could you show last month's decisions, with reasons, to someone who wasn't there?
  • Can you change a threshold without redeploying the agent?

If the answers are short and you're comfortable with them, your DIY guardrails are doing their job. If they aren't, that's the work a runtime authorization layer is for.

For the data side of this, see how to stop an AI agent from exfiltrating customer data. The audit trail is described in the audit log docs, policies as code in the CLI docs, and plans are on the pricing page.

Vulnify decides every sensitive agent action before it runs: ALLOW, REVIEW or BLOCK, with the reasons.

Try it free