Trusted AI Adoption (Part 2): Detection

It’s Monday morning. Your coding agents ran all weekend. Your security dashboard shows the exact same numbers it did Friday afternoon. Same models, the same approved Model Context Protocol (MCP) servers, the same AI assets you are familiar with.

Reassuring.

Then, suddenly, you get a notification: a production deploy failed an audit. The build references a model nobody on your team registered. A second agent pulled an MCP server from a public hub at 2:47 AM on Saturday, while a third hardcoded a call to an external AI service that violated your security perimeter.

Your dashboard is clean, but your supply chain isn’t.

Welcome to the next blog in our series: Detection. This crucial step is where we move beyond assuming your AI supply chain is secure and start proving it.

If you followed Part 1: Consolidation, you learned how to successfully build a single registry for your AI assets to establish the “golden path.” But a golden path only works if agents and developers actually use it. When they don’t, you need a system that catches unapproved components immediately before they become part of your applications.

The Velocity Problem: Agents Don’t Sleep

The fundamental flaw in current security thinking is treating detection as an event. A quarterly audit. A scheduled vulnerability scan. A gate before release.

In yesterday’s Software Development Lifecycle (SDLC), you might have gotten away with periodic scans because humans used to write code at a human pace. But in an agentic supply chain, the velocity fundamentally breaks periodic security.

Coding agents work at lightspeed to autonomously pull dependencies, test external models, and acquire new skills around the clock. Every hour an agent operates autonomously, the gap between what your registry knows and what your development environment actually runs grows wider.

The Failure of “Hopeful” Security

When organizations realize agents and developers are pulling unvetted AI assets, they typically rely on two failing strategies:

1. The Policy Document: Writing a strict policy that states, “All AI models must be pulled from the central registry,” and hoping developers configure their IDEs and agents to comply.
2. The Late-Stage CI Scanner: Plugging a basic scanner into the CI pipeline and hoping it catches everything right before production.

Both methods fail because they wait too long and look in the wrong places. They miss the MCP servers connected directly to local IDEs. They miss the prompt-based agent skills loaded mid-task.

Three Moves to Master Continuous Detection

To move from hope to enforcement, detection must become a continuous, automated discipline. Here is how you build a detection system that actually catches Shadow AI before it catches you.

1. Scan Where the Assets Actually Live

A proxy or gateway only sees traffic passing through it. To find hidden AI assets, you must scan where developers and agents actually put them. Continuous detection requires deep scanning across four critical locations:

• Binaries and Containers: Are models packaged directly inside your Docker images?
• Source Code: Are developers hardcoding API keys to external LLM services in their Python scripts?
• Build Manifests: What external dependencies are being called during the build process?
• Agent Configurations: What skills, plugins, and remote MCP servers are your coding agents authorized to connect to locally?

2. Classify your Binaries

Binary “Approved vs. Rejected” thinking doesn’t work for AI. When you find an asset, you must instantly classify its status to determine the appropriate automated response. Every AI asset in your environment falls into one of four categories:

• Managed: The ideal state. The asset is registered in your central catalog, fully versioned, continuously scanned by security tools (like JFrog Xray), and approved for production.
• Partially Managed: The asset is visible and approved for usage in some projects, but is also found in other projects where it is not allowed to be used.
• Unmanaged (Shadow AI): The asset is completely off the radar. It was pulled directly by a developer or an agent from an external hub (like Hugging Face), bypassing the registry entirely. It is unknown and untrusted.
• Malicious: The asset contains known vulnerabilities, malicious payloads, or violates hard compliance policies, classifying it as an active threat.

3. Make Detection a Continuous Feed

Detection cannot be a snapshot; it must be a continuous feed. Every time an agent pulls a new dependency, every time a developer commits code, and every time a build runs, your detection mechanisms must update your inventory.

This continuous loop ensures that an “Unmanaged” asset pulled on Saturday night is flagged, classified, and either blocked or pushed into the “Managed” workflow by Sunday morning.

The Payoff: From Hoping to Enforcing

Implementing continuous detection fundamentally shifts your security posture:

Stage	Result
Before Continuous Detection	You own the golden path for releases. But when Security finds an unvetted model in last night’s build, you have no answer for whose team pulled it, when, or why. You are the enabler, but you are the last to know what is actually running.
After Continuous Detection	The golden path covers AI. Policies are live, and the registry is the default. Because detection never stops, you know that when coding agents move fast, nothing slips through. When an unmanaged skill lands in a build, you see it first, classify it instantly, and remediate it on the spot without breaking developer velocity.

 

Coming Next: Centralized Visibility and Governance

Detection is the engine that finds the assets. But finding them is only half the battle; you still need to apply policy.

In our next post, we will explore Part 3: Centralized Governance. How to take the continuous inventory your detection tools produce and turn it into a single control plane. We will show you how to enforce security, compliance, and operational policies once, consistently, across every AI asset your developers and agents consume.

Don’t want to wait? Book a live demo to see how the JFrog AI Catalog continuously detects every AI asset across your agentic supply chain and classifies them on the four-state spectrum – enabling you to remediate risks without slowing down innovation.