Intelligence per Microjoule: Why the Next Era of AI Will Be Decided at the Edge

For most of the past three years, the semiconductor industry has measured AI progress in one direction. Models have grown larger, clusters have grown denser, and accelerators have been paired with more high-bandwidth memory with each generation. That build-out has been remarkable, and it will continue. As we look toward 2027, though, the constraints shaping our industry are becoming as important as its capabilities, and the most consequential of those constraints is energy.

The International Energy Agency estimates that data centers consumed roughly 415 TWh of electricity in 2024, about 1.5% of global demand, and projects that figure will more than double to around 945 TWh by 2030, with AI as the primary driver¹. The same build-out has shifted memory supply toward the cloud. Major memory makers have shifted production toward HBM and high-capacity DDR5 for AI data centers, and IDC expects the resulting shortage to persist well into 2027². Counterpoint Research reported that memory prices rose 80 to 90% in the first quarter of 2026 compared with the fourth quarter of 2025³.

Together, these pressures point to the same conclusion: the next phase of AI growth will depend less on concentrating computation in a few places and more on efficiently distributing intelligence across billions of devices. The key metric is shifting from peak TOPS to useful inference per unit of energy.

The Edge is Where the Data Already Lives

IoT Analytics estimates there were about 21.1 billion connected IoT devices at the end of 2025 and forecasts roughly 39 billion by 2030⁴. Most of them are not phones or PCs. They include sensors on factory equipment, wearables on wrists and fingers, medical monitors, thermostats, door locks, and asset trackers. Each produces a continuous stream of data, and very little of it is worth sending anywhere.

This is the core economic argument for edge AI. Moving raw sensor data to the cloud consumes radio energy, bandwidth, and latency, and compromises privacy. Running a small model locally to determine what matters, whether that is an irregular heart rhythm, a bearing beginning to fail, or a spoken command, reduces all four costs at once. The market is responding accordingly. ABI Research forecasts that the edge AI chipset market will grow from US$34.4 billion in 2026 to US$96 billion in 2031⁵.

Much of that revenue will come from larger edge platforms, such as vehicles, smartphones, and AI PCs. The volume story, however, lies at the far edge, in battery-powered and energy-harvesting devices, where the power budget is measured in milliwatts or microwatts and where time between charges often decides whether a product succeeds.

Wearables Show Limitations and Opportunities

The wearables market illustrates both the opportunity and the pressure. IDC forecasts smartwatch shipments to decline 2.8% in 2026, citing memory-related supply constraints that keep prices elevated⁶. FDM CCS Insight points to replacement fatigue and rising component costs as further drags on the category⁷. Smart rings are moving in the other direction. FDM CCS Insight projects ring shipments to grow at a 32% compound annual rate through 2030, reaching about 15 million units⁷.

The form factors gaining share are the smallest, with the least space for a battery, yet consumers expect them to deliver increasingly sophisticated on-device intelligence: sleep staging, recovery scoring, arrhythmia screening, and eventually conversational summaries of long-term health trends. A ring cannot hold a smartwatch-sized battery, and people will not accept daily charging for a device designed to disappear on the hand. Meeting those expectations requires a step change in compute efficiency, not incremental improvement.

Three Technology Shifts to Watch in 2027

1. Supply voltage becomes a primary design lever. For decades, efficiency gains came largely from using smaller transistors. At the leading edge, however, the most advanced node is often the wrong economic choice: wafer costs are higher, and leakage dominates when a device spends most of its life idle. Greater gains come from how circuits operate. Since switching energy scales roughly with the square of supply voltage, running logic and memory near or below the transistor threshold can cut active energy several-fold compared with nominal operation. The trade-off is substantial design complexity involving process variation, timing, and temperature. At Ambiq, we’ve focused on this for more than a decade, and I expect more of the industry to invest in low-voltage design methods, fine-grained power domains, and architectures that keep most of a chip dark most of the time.
2. Memory moves closer to compute and becomes non-volatile. The cloud memory crunch has an edge counterpart. Embedded models are growing, and moving weights between memory and the processor often consumes more energy than the arithmetic itself. Embedded NOR flash is also reaching its scaling limits at advanced nodes. The response is evident across the supply chain. TSMC now has embedded RRAM in high-volume production at 40nm, 28nm, 22nm, and 12nm, with 6nm in development, as well as an embedded MRAM offering. TSMC reports that these memories are widely adopted in edge AI and battery-powered SoCs. Several major microcontroller suppliers already ship products with on-chip MRAM or RRAM⁸. By 2027, I expect emerging non-volatile memories to be a standard part of edge AI architecture planning, with designers focused on minimizing how far data must travel.
3. Software becomes part of the hardware. Efficiency gains in silicon can quickly disappear when a model is mapped poorly. Compilers that plan memory placement, quantize to 8 bits or less, and profiling tools that measure energy per inference are becoming decisive. Teams that successfully ship edge AI products will treat the model, runtime, and silicon as a single co-design problem. Open runtimes such as LiteRT and RTOS ecosystems such as Zephyr are also making it less costly to move models between platforms, shifting competition toward energy efficiency measured in the field.

A supply chain advantage in mature nodes

The memory shortage has highlighted the growing concentration of the AI supply chain. Leading-edge logic, advanced packaging, and HBM capacity are scarce, with supply allocated first to the highest-margin customers. Far-edge silicon, by contrast, is largely produced on mature and specialty process nodes, which have broader foundry availability and longer production lifecycles. 

For medical and industrial OEMs that need multi-year supply commitments, stability is an important design consideration. Designers can further limit their exposure to the DRAM market by selecting architectures with enough on-chip SRAM and non-volatile memory to run their models without external DRAM.

Looking toward 2027 and beyond

Several developments seem likely over the next few years.

Hybrid AI becomes the default architecture— Small always-on models will run at the sensor, mid-sized models on the phone or gateway, and the largest models in the cloud. The far edge will act as a filter, deciding which events warrant expensive computation, making the entire system, including the data center, more efficient.

Energy per inference becomes a published specification— Battery life has long been a key consumer metric. As AI features become standard in small devices, product teams and buyers will increasingly compare chips by energy per inference on representative workloads. Benchmarks that report energy alongside latency will carry more weight in purchasing decisions.

Physical AI expands the far edge— Robotics, industrial automation, and smart buildings rely on many distributed sensing nodes, not just a central processor. Each node could host local intelligence, and many will run on batteries or harvested energy.

Regulation raises the bar on power— Beginning in 2027, the EU Battery Regulation will require batteries in many portable products to be removable and replaceable by end users, with some exemptions⁹. Together with sustainability reporting and right-to-repair efforts, these rules make designs that last longer on smaller batteries more valuable.

The past few years have shown what AI can do when energy and memory seem nearly unlimited. The years ahead will depend on making intelligence energy-efficient and affordable wherever data is created. That work happens at the level of the transistor, the memory cell, and the compiler—and it will determine how deeply AI reaches into everyday life.

Sources

1. International Energy Agency, Energy and AI, April 2025. iea.org
2. IDC, "Global Memory Shortage Crisis: Market Analysis and the Potential Impact on the Smartphone and PC Markets in 2026" (idc.com) and "Higher ASPs, lower unit volumes: How the memory crisis is reshaping the PC and smartphone outlook" (idc.com), 2026.
3. Counterpoint Research, "Memory Prices Surge Up to 90% From Q4 2025," Memory Price Tracker, February 2026. counterpointresearch.com
4. IoT Analytics, "State of IoT 2025: Number of connected IoT devices growing 14% to 21.1 billion globally," October 2025. iot-analytics.com
5. ABI Research, "Four Edge AI Trends That Semiconductors Should Know from ABI Research's 2Q 2026 Market Update," June 15, 2026. abiresearch.com
6. IDC, Worldwide Wearable Device Tracker, "Worldwide Wearable Device Market, 2026Q1." idc.com
7. FDM CCS Insight, "Global Wearables Market to Decline in 2026 as Demand for Smartwatches Slows," July 17, 2026. ccsinsight.com
8. TSMC, "Embedded Non-Volatile Memory (eNVM) Technology." tsmc.com
9. Regulation (EU) 2023/1542 of the European Parliament and of the Council concerning batteries and waste batteries, Article 11. eur-lex.europa.eu