Neural Rendering is coming to DirectX – Microsoft Confirms
Microsoft’s investing in Neural Rendering, and that means big changes are coming to DirectX
Following the reveal of Nvidia’s RTX 50 series of graphics cards, Microsoft has confirmed that it is bringing Neural Rendering features to its DirectX API. Neural Rendering covers a broad range of new techniques that leverage Artificial Intelligence and Machine Learning (AI and ML) to alter traditional graphics pipelines. These changes will place 3D graphics on the road of transformation.
The start of Microsoft’s enablement of Neural Rendering in DirectX starts with support for Cooperative Vectors. This addition will unlock the power of “neural shading” on Nvidia RTX 50 series GPUs. These are the first Nvidia GPUs to feature Tensor Cores with Neural Shading capabilities.
Neural Rendering: A New Paradigm in 3D Graphics Programming
In the constantly advancing landscape of 3D graphics, neural rendering technology represents a significant evolution. Neural rendering broadly defines the suite of techniques that leverage AI/ML to dramatically transform traditional graphics pipelines. These new methods promise to push the boundaries of what’s possible in real-time graphics. DirectX is committed to cross-platform enablement of neural rendering techniques, and cooperative vectors are at the core of this initiative.
We are excited to share our plans to add cooperative vector support to DirectX, which will light up cross-platform enablement of the next generation of neural rendering techniques.
Cooperative Vectors will accelerate real-time AI workloads for real-time rendering. It enables the multiplication of matrices by arbitrarily sized vectors, optimising matrix-vector operations. Cooperative Vectors also allow AI tasks to run at different stages of shader pipelines, allowing small neural networks to operate within pixel shaders without occupying the whole GPU. This allows neural graphics techniques to be implemented without stalling the graphics pipeline every time an AI/ML operation is used.
To put it simply, Microsoft is changing DirectX to make AI part of the graphics pipeline, not a separate component that needs to be addressed separately. This gives new options to developers, allowing them to use AI to enhance various aspects of game rendering.
What are Cooperative Vectors, and why do they matter?
Cooperative vector support will accelerate AI workloads for real-time rendering, which directly improves the performance of neural rendering techniques. It will do so by enabling multiplication of matrices with arbitrarily sized vectors, which optimize the matrix-vector operations that are required in large quantities for AI training, fine-tuning, and inferencing. Cooperative vectors also enable AI tasks to run in different shader stages, which means a small neural network can run in a pixel shader without consuming the entire GPU. Cooperative vectors will enable developers to seamlessly integrate neural graphics techniques into DirectX applications and light up access to AI-accelerator hardware across multiple platforms. Our aim is to provide game developers with the cutting-edge tools they need to create the next generation of immersive experiences.
Note that Nvidia’s RTX 50 series GPUs are currently the only GPUs to support Neural Shaders. However, Microsoft is working with AMD, Intel, Nvidia, and Qualcomm to enable cross-vendor support for these new features. Microsoft wants these changes to be a core part of DirectX.
What’s Next For Neural Rendering?
The HLSL team is working with AMD, Intel, NVIDIA, and Qualcomm on bringing cross-vendor support for cooperative vectors to the DirectX ecosystem. Stay tuned for more updates about cooperative vectors and its upcoming Preview release!
You can join the discussion on Microsoft’s new DirectX Neural Rendering features on the OC3D Forums.
