The US Cybersecurity and Infrastructure Security Agency (CISA) and National Security Agency (NSA), published the interagency guidance, "Careful Adoption of Agentic AI Services," in collaboration with cybersecurity agencies from Australia, Canada, New Zealand, and the UK. The guidance highlights risks unique to agentic AI systems and best practices for mitigating such risks.

The guidance focuses on agentic AI systems composed of multiple AI agents working in concert, each agent having its own LLM, input stream, tools, permissions, external data sources, memory, and output stream. AI agents are capable of analyzing and reasoning over the agents' environment and context to make decisions about actions to execute within its environment (e.g. writing/editing files, filling forms, pressing buttons, entering commands, writing code, etc.). 

A key risk of tool use is malicious actors publishing malicious tools with false and persuasive descriptions, leading a privileged AI agent to execute malicious code on the systems it has access to. The risk of malicious tools is exacerbated where agents are able to dynamically install additional content to enhance the agent's own capabilities.

AI agents are also capable of creating additional agents or "spawning sub-agents." A risk unique to agentic AI systems is that where one agent is compromised, it can have a cascading effect on other agents or sub-agents within the same agentic system, which means that a compromised low-permission agent could affect an agent with higher levels of system access, where the higher-privileged agent is receiving the affected agent's output as its input.

Multi-agent systems are much more complex than traditional LLM chatbots, increasing the difficulty of tracing the reasoning for why an agentic system took a particular action. The guidance suggests consolidating individual agent logs into human-readable logs for the entire agentic system. The guidance also suggests including reasoning traces and tool use in logs, and considering what artifacts produced by agents are available for reporting.

Best practices include—

• Context control: Control the agents' context by grounding the agent in user-provided, relevant contextual information using retrieval augmented generation (RAG) and prompt engineering to mitigate hallucinations.
• Human in the loop: Implement human control points throughout AI workflows. Do not simply delegate to AI the decision of when it should check in with human operators.
• Permissions & identity management: Implement identity management, identity-based boundaries/permissions, and segmentation, to prevent AI systems from having more access than needed for the AI use-case it fulfills. Prohibit agents from modifying their own privileges.
• Graduated testing in separate environment: Run tests and simulations before deploying AI agents or agentic systems in production environments. Confirm the privileges of the agents. Use a phased/graduated approach to increasing access or autonomy/independence, to keep access limited to the least privilege necessary.
• Filtering out malicious prompts: Place filters between the user and the AI, such as prompt injection filters, so that inputs are validated and sanitized before an AI can receive the instructions and execute.
• Maintain a list of trusted/permitted third-party tools: Maintain a registry of trusted third-party components and do not use components that are not on your organization's approved list.
• Consensus controls: Implement consensus controls where multiple agents must agree on a decision before an action can be executed, where the risk level warrants. Combine this with human-in-the-loop for higher risk agentic systems.

Check out the complete interagency guidance if your organization is exploring agentic workflows.