AI Agents Fail When Architecture Is Wrong

When AI agents underperform or fail in business settings, companies often blame the AI models themselves or human resistance. However, the real issue lies deeper: traditional enterprise architectures are not built to handle agentic workloads. According to a comprehensive analysis from Salesforce, successful AI agent deployment requires a complete rethinking of how systems are structured, moving beyond simply adding agents to existing frameworks. This insight shifts the conversation from model capabilities to architectural design, highlighting that intelligence alone is insufficient without the right infrastructure to support it.

The researchers identified that an effective Agentic Enterprise relies on four interconnected architectural layers: the system of engagement where people and agents collaborate, the system of agency comprising the agents themselves, the system of work which includes business applications, and the system of data representing unified enterprise information. The success of AI agents depends on how seamlessly these layers operate together. Drawing from extensive experience in strategy and generative AI, the team developed eight design pillars that serve as a framework for implementation and highlight common pitfalls. This approach emphasizes that architecture, not just AI models, determines whether agentic systems thrive or fail.

One critical pillar is modular workability, which the paper compares to building with Lego blocks. Companies cannot afford to rebuild entire architecture components for each new agent function, as this process is time-consuming, costly, and leads to ungovernable systems. Instead, organizations should work from a menu of modular, premade items. For example, if a company develops a customer onboarding agent, it should be able to reuse the same data connectors and integrations when creating a customer offboarding agent six months later. Problems arise when even minor variations require complete architectural overhauls, indicating a flawed design that hinders scalability and efficiency.

Another essential element is unified data with metadata, ensuring agents understand context beyond raw information. While humans can recognize that Acme Corp in a CRM system is the same as Acme Corporation in an ERP, agents need metadata, business glossaries, and ontologies to make these connections. Without proper data interconnection, agents may fulfill tasks incorrectly, such as increasing sales through deep discounts without realizing the company loses money on each sale. The framework advocates for consistent labeling and access to contextual tools so agents can learn and make financially sound decisions over time.

Governance and trust must be embedded into the architecture's DNA, with unified observability providing real-time visibility into agent actions, reasoning, context, compliance, and business outcomes. This observability spans IT and business domains, allowing organizations to track which actions are most used, identify mistakes, and optimize performance. Without it, tracing problems becomes impossible when errors occur. Governance should include distinct agent identities, time-limited task-based permissions, and workflow checks against established policies. The paper warns that if trust is lacking, adoption will fail, as agents often handle sensitive tasks like writing to financial systems or approving contracts.

Strategic human oversight is also crucial, likened to airport security screening where only suspicious items receive thorough examination. Agents should handle routine, low-risk tasks by default, alerting humans when requests fall outside their domain or when users ask to speak with a person. Smooth handoffs with context prevent humans from restarting cases. Issues arise when human intervention is required for every decision or when agents operate completely unsupervised, both extremes undermining efficiency and safety. This balance ensures agents enhance productivity without compromising control.

Real-time connectivity and resilience are non-negotiable, as business speed demands agents that are always on and responsive across channels like texts, emails, calls, or APIs. The architecture must handle unpredictable computing needs, with AI-ready infrastructure that scales based on workload spikes and distributed loads to prevent catastrophic failures. Without this, systems become brittle, risking outages during increased agent reasoning. Additionally, interoperability is key, as no single vendor can provide everything for agentic AI. An open architecture with standard interfaces, protocols, and portable workflows allows easy integration of new models and data sources, avoiding vendor lock-in that could hinder future scaling.

These eight principles serve as control surfaces for scaling AI agents while maintaining trust, enabling organizations to defend their systems to regulators, customers, and boards. Without them, companies are left with disconnected experiments rather than a cohesive Agentic Enterprise. The framework offers a realistic path forward: phased, bounded implementations focused on closing the gap where most companies struggle. By prioritizing architectural design over model selection, businesses can transform intelligence into actionable, reliable systems that drive meaningful outcomes.