Is Microsoft making NPUs useless? – Windows 11’s Local AI is getting GPU support

Microsoft is giving Windows 11’s Local Language AI Models official Nvidia GPU support

Microsoft is adding support for Windows 11’s Local Language (AI) models to non-Copilot+ PCs. With support for GPU hardware, Windows 11’s AI features will no longer be locked to systems with “NPU” (Neural Processing Unit) hardware, effectively killing everything that made NPU hardware special.

With Microsoft’s Windows App SDK 2.2 Experimental 9 update, as spotted by Windows Latest, Windows 11’s language model APIs will run on non-Copilot+ PCs with specific GPUs. Right now, this is limited to Nvidia RTX 30 series GPUs (or newer) with at least 6GB of VRAM. In time, this support could come to AMD and Intel GPUs.

NPU VS GPU

NPU Hardware is designed to run AI models and specialises in efficiency over raw compute performance. GPUs are heavy-duty parallelised processors that are the home of serious AI. There’s a reason why AI datacenters mostly use Nvidia’s GPU hardware rather than dedicated NPUs. That said, Microsoft believed that NPUs were the future when it created its Copilot+ program.

Copilot+ PCs require 16GB of system memory, SSD storage, and an NPU that’s capable of 40 TOPS of AI performance. With dedicated GPU support for Windows 11’s Local Language models, the utility of dedicated NPU hardware may soon be lost. While dedicated NPU hardware could have its benefits, with dedicated GPU support, PC manufacturers may be better off using an NPU’s silicon to make their integrated GPUs larger.

What’s new in WinAppSDK 2.2 Experimental 9:

• Language Model APIs on GPU [Experimental]. The Language Model APIs now run on non-Copilot+ PCs equipped with a supported GPU, bringing local language model capabilities to a broader range of Windows 11 devices. Supported hardware includes NVIDIA GeForce RTX 30 series and newer with 6+ GB vRAM. GPU inference requires Developer Mode to be enabled and a Windows Insider Experimental Channel build. The GPU model is not pre-installed; it is downloaded on demand via EnsureReadyAsync through Windows Update. Apps should check GetReadyState and display a consent dialog before triggering the download. Users can manage the model at Settings > System > AI Components. For responsible AI guidance, see the new Transparency Note: Language Model APIs on Non-Copilot+ PCs.

– Microsoft

Copilot+ Software for non-Copilot+ PCs

In terms of raw power, GPUs have always been faster than NPUs for local AI workloads. Initially, Microsoft limits its Copilot+ features to PCs with dedicated NPU hardware. That may soon change. This will give Windows 11 PCs with supported GPUs access to features like text-to-image, text generation, Windows Recall, and more.

If Copilot+ features are coming to non-Copilot PCs, Microsoft’s Copilot branding could effectively be dead. So could the need for dedicated NPU hardware inside Windows PCs.

Will NPUs soon become useless silicon?

Many PC enthusiasts have long called NPUs “useless silicon”. For most workloads, this silicon did nothing useful, making it a waste of space. Silicon is expensive, and wasted space is something that hardware makers should avoid at all costs. If Microsoft allows GPUs to handle its dedicated NPU workloads, the need for NPU hardware diminishes. Why not just build a larger GPU and run your AI tasks on it? At least that way the hardware will have more utility. GPUs can be used for more tasks than NPUs after all.

While Microsoft’s GPU support for its AI features is coming, it does not guarantee that NPU hardware is going away. That said, with such diminished utility, it’s hard to see manufacturers investing heavily in their NPU hardware. If full GPU support arrives for AMD, Intel, and Nvidia GPUs, it is possible that NPU hardware could be abandoned in a few hardware generations.

You can join the discussion on Microsoft’s plan to bring its Copilot features to non-Copilot PCs on the OC3D Forums.