Beyond Faster Chips: Why Data Movement Will Shape AI’s Next Chapter in 2027

For most of the past decade, the semiconductor industry has answered AI's appetite for compute by just building more of it. More cores, more accelerators, more transistors, more advanced process nodes. It worked, and it worked remarkably well. But heading into 2027, with the continuing growth in model size, the exploding usage, the data needed, that playbook is running out of road.

The reason is simple. Compute is only useful when it has data to work on. An accelerator with enormous peak throughput that sits waiting on memory, or data from another accelerator, is just expensive silicon at idle, that is still a huge energy and cost sink. Adding more accelerators doesn't fix that. It often just adds more traffic to an already congested system.

Moving data to processors, between components, and across increasingly complex systems now matters as much as the compute itself. As AI workloads grow larger and more varied, the industry can no longer treat that as a secondary concern.

Three shifts will define how the industry responds in 2027: performance measured at the system level, chiplet architectures that raise the stakes on integration, and hardware designed around the actual flow of data.

Performance at System (Rack) Scale

The industry has long marked progress with processor-level numbers: clock speeds, transistor counts, peak computational throughput, and so on. They're still useful. They just don't tell you how a system will behave under a real workload, and AI is exposing that gap.

Training moves enormous volumes of data across processors, memory, and networking. Inference brings its own pressures as applications become more interactive, multimodal, and reasoning-heavy. Some workloads are compute-bound. Majority however are bound by memory bandwidth, latency, or inter-accelerator communication. The bottleneck shifts with the workload, which means the definition of performance has to shift too.

Higher theoretical throughput means nothing if the surrounding architecture can't keep the processor fed. More memory capacity or bandwidth delivers little if the interconnect fabric adds congestion or unnecessary transfers. Either way, compute starves.

Energy availability makes this issue sharper. The International Energy Agency's 2026 analysis of energy and AI projects that global data center electricity consumption will climb from about 485 terawatt-hours in 2025 to 950 terawatt-hours by 2030, with AI-focused data center consumption tripling over the same period. Demand for compute can keep growing. Power availability, cooling capacity, and infrastructure budgets can't.

So, in 2027, the metrics that matter will be useful work delivered per watt and per dollar, not peak compute in isolation. For inference, that means tokens per second, tokens per watt, and cost per token. The architectures that win will keep compute productive while cutting unnecessary data movement, latency, and energy. The more you can do in each chip package, the closer you can get the real compute-related data to the element, the greater the efficiency.

Chiplets can liberate… or constrain

Chiplets let designers combine smaller, specialized dies rather than integrating every function onto one. That offers real advantages: reuse of proven IP; use of best-in-class elements; higher-yield, cost-effectiveness from fit-for-purpose implementation; manufacturing flexibility; and workload-specific optimization. But it also changes the problem.

Components that once communicated on a single die have to instead talk across die boundaries, and every boundary adds interfaces, communication overhead, power, and coordination. A modular architecture doesn't eliminate the data movement problem. It multiplies it.

Picture an AI platform that combines general-purpose processors, specialized accelerators, high-bandwidth memory, and networking. Each piece can be individually optimized, and the platform can still underperform, because its performance depends on how well those pieces exchange information. When communication becomes the bottleneck, the benefits of specialization evaporate.

That's why interoperability matters so much. The Universal Chiplet Interconnect Express (UCIe) Consortium released its 3.0 specification in August 2025, supporting data rates of 48 and 64 GT/s and improving bandwidth density, power efficiency, and system manageability. It's a meaningful step toward scalable multi-die systems, but a standard interface solves only half the problem.

Connecting chiplets is one challenge. Making them work together efficiently is another. Designers still must decide how traffic flows between components, how memory is shared and accessed, how coherency is maintained where it's needed, and how the architecture adapts as workloads change. The IEEE's 2026 Heterogeneous Integration Roadmap reinforces the point: design, manufacturing, packaging, and reliability now need to be coordinated across the whole system, not optimized in silos.

Expect a stronger push toward reusable architectures that stretch across products and markets. An edge AI device and a large data center have very different power, latency, and cost targets, but both benefit from shared architectural principles and reusable building blocks. The opportunity is configurability without starting every design from scratch, and that requires thinking beyond the chiplets themselves to the architecture that ties them together.

Data Movement Becomes a First-Class Design Decision

The most important change heading into 2027 is that designers will address data movement earlier.

In complex AI systems, data movement decisions shape nearly everything else: latency, bandwidth, power, memory utilization, optimal partitioning, and ultimately, how much useful work the processors can actually do. More importantly, it impacts how you can deliver a system that is orders of magnitude better in tokens/watt and tokens/dollar to ensure economic viability.

Take a system where several accelerators share memory and other resources. Its effectiveness depends on the bandwidth at each interface and on how the system communicates. Where is data generated? Where does it need to go, and how often? Which transfers can be eliminated entirely? What happens when components contend for the same resources?

The answers depend on the workload. Training demands frequent communication and synchronization across many accelerators. Inference prioritizes memory-access efficiency, predictable latency, and throughput under fluctuating demand. Automotive and industrial edge systems need strict real-time behavior in a fraction of the power envelope. No single interconnect topology or memory architecture is optimal for all of them.

That's why the industry will lean harder on approaches that let architects evaluate data movement early, explore more configurations, and understand system-level trade-offs before committing to implementation. Software-driven architecture exploration is especially valuable here. Rather than treating the interconnect as whatever falls out of processor and memory selection, architects can model traffic, evaluate bandwidth and latency requirements, and optimize communication as an integral part of the design.

That doesn't make modern chip development simple. But it surfaces architectural limitations while they're still practical to fix. A limitation discovered late in the design cycle is a limitation you have to live with.

The stakes extend well beyond the data center. Intelligent computing is spreading into vehicles, industrial systems, robotics, and consumer devices. At every scale, designers face the same fundamental challenge: getting the right data to the right place at the right time, with as little overhead as possible. That holds no matter which processor architecture, accelerator technology, or manufacturing process wins out.

What This Means for 2027

None of these challenge make advances in processors or manufacturing less important. Continued innovation in compute, memory, and packaging is essential. But better components don't automatically add up to better systems.

The next generation of AI infrastructure will be more heterogeneous: processors, accelerators, memory technologies, and interconnects combined in new ways and scaled across multiple dies, packages, and entire racks. At the same time, designers face mounting pressure to rein in power, control development costs, and ship differentiated products faster.

Those pressures pull against each other. Higher peak bandwidth can mean higher power. Lower latency can mean more implementation complexity. Greater modularity can improve reuse while demanding tighter coordination. There are no free wins, only trade-offs made well or made badly.

In 2027, I expect the conversation to shift from who builds the fastest individual chip to who builds the most efficient, scalable, and adaptable system.

The industry has spent decades making processors extraordinarily capable. Now it has to become just as sophisticated about how they communicate. The question is no longer how much compute we can build. It's how much of it we can actually put to work.