Skip to main content

Agentic AI & AI Agents

Last updated on by kay-schmitteckert

Disclaimer

The Agent Gateway is not yet generally available (GA). As a result, the current architecture supports unidirectional (outbound) communication only.

This reflects a transitional state - key components enabling full bidirectional capabilities are expected to be released soon and will evolve the architecture accordingly.

AI agents represent a new paradigm in enterprise software, combining large language models (LLMs) with tools, memory and reasoning capabilities to autonomously execute complex, multi-step tasks. In the SAP ecosystem, AI agents bridge the gap between intelligent automation and core business processes, enabling systems to dynamically adapt, reason and act in real-time.

This reference architecture provides a comprehensive guide to developing, deploying and managing AI agents in your SAP ecosystem powered by SAP Business Technology Platform (BTP). It details the architectural patterns, components and best practices for building both low-code and pro-code agents, integrating them with Joule through bidirectional A2A communication and ensuring seamless interoperability across the enterprise landscape.

From automating routine workflows to enabling sophisticated analytics on structured data and extending digital processes into the physical world with embodied AI, this guide covers the full spectrum of agentic capabilities within and beyond your SAP ecosystem.

Architecture​

The core architecture for AI agents at SAP is based on the AI Foundation, providing the services for building, running and managing intelligent agents across the enterprise landscape.

image of solution diagram
Copy to clipboard
Solution Diagram Resources
You can download the Solution Diagram as a .drawio file for offline use. Alternatively, you may view and edit the Solution Diagram directly on draw.io.
Please note that any changes made online will need to be saved locally if you wish to keep them.
DownloadEdit

The architecture centers around Joule as the orchestrator, with custom agents on SAP BTP connected to SAP Cloud Solutions and 3rd Party Systems:

  • Joule: Central AI copilot providing unified user interface and orchestration. Routes requests to agents and skills, manages conversations and enables bidirectional A2A communication through the Agent Gateway (inbound) and Joule Capabilities (outbound).

  • Custom Agents on SAP BTP: Two development approaches - Low-Code via Joule Studio with visual configuration and automatic Joule registration and Pro-Code using SAP Cloud SDK for AI with frameworks (LangGraph, AG2, CrewAI, Smolagents) integrated via A2A protocol.

  • Generative AI Hub: Foundation models, prompt optimization, orchestration capabilities (grounding, templating, data masking, I/O filtering) and vector search via SAP HANA Cloud.

  • SAP Cloud Solutions: Seamless integration with SAP S/4HANA, SuccessFactors, Business Data Cloud, Concur, Customer Experience and Business Networks.

  • 3rd Party Integration: External (AI) platforms (Google Cloud, Microsoft Azure, AWS, IBM Cloud) and (AI) clients consume Joule Agents via the Agent Gateway or are orchestrated by Joule. MCP Servers and APIs provide tools and data sources.

  • Security: SAP Cloud Identity Services manages authentication, authorization and identity federation across all connections.

Development Approaches​

SAP supports two complementary paths for building AI agents, each optimized for different development needs and complexity requirements.

Low-Code with Joule Studio: Build agents through visual configuration in SAP Build for rapid development and deployment. Joule Studio provides configuration-driven orchestration with multi-step reasoning, RAG capabilities and seamless SAP integration. Agents automatically register with Joule and run on a managed runtime. See Low-Code AI Agents with Joule Studio.

Pro-Code with SAP Cloud SDK for AI: Develop agents using Python or TypeScript with full programmatic control. Integrate popular frameworks like LangGraph, AG2, CrewAI and Smolagents with CAP for orchestration. Agents connect via A2A protocol for maximum flexibility in complex scenarios. See Pro-Code AI Agents on SAP BTP. Specialized use cases: AI Agents for Structured Data and Embodied AI Agents.

Integration Patterns​

AI agents integrate with Joule through bidirectional A2A communication, enabling seamless orchestration and external consumption.

Inbound Integration (Agent Gateway): External systems consume Joule Agents through a publicly accessible gateway supporting synchronous and asynchronous A2A patterns. Third-party platforms like Google Vertex AI, Microsoft Copilot Studio and AWS Bedrock can delegate SAP-specific tasks to Joule Agents. See Integrating Joule Agents into Your Ecosystem.

Outbound Integration (Bring Your Own Agent): Joule orchestrates external code-based agents built with any A2A-compliant framework. Supports synchronous communication, asynchronous callbacks for long-running tasks and multi-turn conversations with context handling. See Integrating AI Agents with Joule.

Open Standards Strategy: SAP embraces A2A as the preferred standard for multi-agent collaboration and uses MCP internally for semantic tool connectivity. See Agent & Tool Interoperability.

Services and Components​