Avasant’s Key Takeaways from AMD Advancing AI 2026

As enterprise AI adoption accelerates, demand for the infrastructure required to support these workloads is increasing rapidly. The growing deployment of foundation and large language models across business processes is driving an unprecedented surge in demand for inference.

The rise of agentic AI is fundamentally reshaping the nature of inference workloads. Unlike traditional AI requests that typically involve a single model invocation, AI agents decompose objectives into multiple tasks, perform repeated inference, retrieve enterprise knowledge, invoke tools and APIs, execute code, and iteratively evaluate outcomes. As a result, a single agentic workflow can require significantly more compute, memory, and data movement than conventional inference, depending on the complexity of the task and the number of steps involved.

While the underlying technology stack remains largely unchanged, the balance of infrastructure investment is evolving. As agentic workloads become more sophisticated, CPUs play a larger role in orchestration and tool execution, AI accelerators handle repeated model inference, high-bandwidth memory supports larger context windows and models, high-speed networking enables data movement across distributed systems, and AI software platforms coordinate, optimize, and govern increasingly complex multi-agent workflows.

From Avasant’s perspective, these developments reflect a shift in enterprise AI infrastructure planning, where organizations must consider how different infrastructure components support the execution of increasingly complex AI workloads. The announcements at AMD Advancing AI 2026 provide examples of how infrastructure providers are responding to these evolving requirements.

Changing Infrastructure Requirements for Agentic AI

These changes in inference workloads have significant implications for how enterprises design and operate AI infrastructure. As AI tasks place varying demands on compute, memory, and data movement, infrastructure requirements differ substantially across workloads. The challenge is ensuring infrastructure can efficiently accommodate this variability as AI adoption scales. Three challenges stand out:

1. Variation in Resource Requirements

Agentic workloads impose highly variable demands on compute and memory, depending on the model, context window, and the number of inference steps required to complete a task. A simple customer service query may require minimal resources, while a complex research workflow can involve extensive context and repeated inference. This makes it difficult for enterprises to accurately translate AI demand into infrastructure requirements, particularly as large numbers of workloads execute concurrently.

2. Data Movement Bottlenecks

Agentic workflows involve repeated data transfers between accelerator memory, host memory, databases, enterprise applications, and external services. These transfers occur during information retrieval, tool invocation, and the exchange of intermediate results between workflow steps. Depending on data access patterns, transfer frequency, and system architecture, moving data between these components and processing retrieved information can introduce significant latency, potentially becoming a bottleneck even when sufficient accelerator capacity is available. Efficient data movement and the reduction of unnecessary data transfers are therefore important for minimizing end-to-end execution time in complex, data-intensive AI workloads.

3. CPU-Accelerator Coordination Bottlenecks

Agentic workflows involve multiple stages of execution, including model inference, task orchestration, information retrieval, and tool invocation. While accelerators execute model inference, CPUs handle orchestration, retrieve information from enterprise applications, execute tool calls, and prepare inputs for subsequent inference steps. When these CPU-driven operations must complete before the next inference step can begin, accelerators may remain idle while waiting for the next task. Unless the available capacity can be used by other concurrently scheduled workloads, these gaps can reduce accelerator utilization and affect the compute resources required to support agentic workloads.

The AMD Response to Changing Enterprise AI Infrastructure Needs

AMD is extending its AI infrastructure stack across CPUs, GPUs, networking, edge, and software to address the changing resource requirements of increasingly complex inference workloads. These capabilities address different aspects of AI infrastructure, from host-side processing and model execution to accelerator connectivity and workload optimization.

1. A Workload-Specific CPU Architecture for Agentic AI

A key development in AMD’s 6th Gen EPYC™ “Venice” portfolio is its focus on workload-specific CPU configurations for agentic AI infrastructure. AMD has outlined three distinct roles: executing agents in sandbox environments, serving as GPU host nodes, and running general-purpose applications. Venice addresses these roles through different combinations of core density, frequency, memory bandwidth, and I/O capabilities.

For agent CPU servers, high-core-count configurations of up to 256 cores and 512 threads support concurrent agent execution. For GPU host nodes, AMD highlights high-frequency EPYC configurations with high CPU-to-GPU bandwidth and PCIe Gen 6 connectivity to support accelerator-based workloads.

This differentiation reflects a shift from uniform server configurations toward CPUs tailored to specific workload roles. For enterprises, it provides greater flexibility in provisioning CPU resources for agent execution and GPU hosting, while enabling more targeted infrastructure planning as agentic AI deployments scale.

2. AMD Instinct™ MI455X for Large-Scale AI Workloads

AMD’s Instinct™ MI455X GPUs offer higher memory capacity, bandwidth, and compute capabilities than the previous generation MI355X, meeting the demands of increasingly large and complex AI workloads. The MI455X offers 432 GB of HBM4, up 50% from the MI355X’s 288 GB of HBM3E, while peak memory bandwidth increases from 8 TB/s to 23.3 TB/s, approximately 2.9 times higher. AMD also cites up to four times higher peak theoretical MXFP4 performance.

The expanded memory capacity enables larger models and longer context windows to fit within accelerator memory, while higher bandwidth and low-precision compute support faster data access and AI processing. These capabilities give enterprises greater flexibility in deploying memory- and compute-intensive workloads, although realized performance gains will depend on model architecture, workload characteristics, and system configuration.

3. Building the Networking Foundation for Rack-Scale AI

As AI infrastructure moves toward rack-scale deployments, networking must support communication within accelerator systems while also handling infrastructure services across the rack. AMD’s architecture combines UALink for scale-up connectivity with Pensando data processing units (DPUs), which help secure network traffic and accelerate software-defined networking (SDN) and storage operations.

This reflects an important architectural consideration: scaling AI infrastructure requires networking and infrastructure services to keep pace with increasing compute density. By integrating accelerator connectivity and offloading selected networking, storage, and security functions to DPUs, AMD incorporates these capabilities into its rack-scale design rather than treating networking as a standalone component.

4. Coordinating Agentic Workloads Through AMD’s Software Stack

AMD’s software strategy extends beyond optimizing individual inference workloads to supporting the development and coordination of agentic AI. ROCm.AI provides the software foundation for AI workloads, with HyperLoom analyzing inference performance, identifying bottlenecks, generating optimizations, and testing changes. Agentic Mesh coordinates and orchestrates agentic workloads, while the Agentic AI Sandbox supports the development and experimentation of agentic applications and workflows.

These capabilities address distinct requirements in operationalizing agentic AI. While inference optimization improves the use of available compute resources, agent coordination supports the execution of multistep workflows, and the sandbox provides an environment for developing and testing them. For enterprises, the implication is that scaling agentic AI requires software capabilities not only to optimize model execution, but also to coordinate agents and support the iterative development of workflow.

What’s Next for Enterprise AI Infrastructure

Three developments will shape the next phase of enterprise AI infrastructure:

1. Training infrastructure for custom models: Enterprises are starting to build and adapt smaller, specialized models for specific workloads instead of relying on a general-purpose model for every task. River AI, for example, provides a platform for fine-tuning and reinforcement learning on open models, enabling organizations to create and serve models tailored to their requirements. This adds model training and post-training workloads to an infrastructure environment where inference demand is already growing rapidly.
2. Sovereign AI infrastructure: Governments and enterprises are investing in locally controlled AI compute, data, models, and software to address data governance, security, and regulatory requirements. AMD and TCS, for example, are developing a Helios-based infrastructure blueprint in India designed to scale to 200 MW.
3. Physical AI infrastructure: As AI moves into factories, warehouses, and robotics, infrastructure will need to support real-time AI processing close to the machine. AMD’s Kria AI platform, for example, combines CPU control, GPU and NPU inference, and unified memory in an edge system designed for autonomous robotics. This brings AI compute into environments where power, space, latency, and connectivity are much more constrained than in a data center.

Taken together, these developments show why enterprise AI infrastructure is becoming a full-stack and multilocation challenge. From Avasant’s perspective, enterprises will increasingly need to plan not only for accelerator capacity, but also for the CPU, memory, networking, software, and deployment environments required by each AI workload.