Microsoft’s DirectML is the next-generation game-changer that nobody’s talking about
Microsoft’s DirectML is the next-generation game-changer that nobody’s talking about
We’ve spoken about Microsoft’s DirectML before at OC3D, but as a whole, DirectML has received little attention from the media. DirectML isn’t a confirmed feature of Microsoft’s next-generation console, but the timing of DirectML’s development suggests that hardware support is planned for the Xbox Series X.
As many of you know, Microsoft revealed its DXR API in 2018, bringing the world’s attention to realtime raytracing. At that time, most people dismissed DirectX Raytracing (DXR) as a feature which won’t be ready for the release of Microsoft’s next-generation console. Now, DXR support has been confirmed for Microsoft’s Xbox Series X, and AMD has confirmed that new Radeon hardware will support hardware-accelerated DXR raytracing in 2020, making DXR a reality for next-generation games.
Microsoft revealed DirectML at the same time as DXR, but DirectML didn’t receive much attention. Companies are expected to talk about AI these days, and at the time, it was hard to judge the potential of AI within the gaming market. Raytracing has obvious benefits, but AI does not. Enter Nvidia and its Tensor cores.
Nvidia’s DLSS (Deep Learning Super Sampling) technology is the first attempt from a major GPU manufacturer to leverage the power of AI to improve the PC gaming experience. Nvidia uses DLSS to upscale rendered images to higher resolutions, allowing games to run better at higher resolutions by upscaling a lower resolution image. A game could be rendered at 1440p and upscaled to 4K to deliver incredible performance increases to gamers, making higher resolution gaming a lot more accessible (when playing supported titles). While the feature has its downsides (degraded image quality), new implementations of DLSS are difficult to distinguish from native resolution renderings, revealing the potential that AI has within the PC market.
DirectML – The API that could bring DLSS-like features to everyone
While DirectML hasn’t been confirmed as a next-generation console feature, you can be sure that Microsoft has been considering the option heavily. Work on DirectML has been happening, at least publically, for as long as DXR has, making it likely that AMD is working on hardware DirectML support for its next-generation graphics cards.
Microsoft has already showcased the potential of machine learning in gaming applications, with the image below showcasing what happens when Machine Learning is used to upscale an image to four times its original resolution (basically from 1080p to 4K) to generate a sharper final image and reduced aliasing. The image below is a comparison between ML Super Sampling and bilinear upsampling.
This technique has also been showcased during one of Microsoft’s SIGGRAPH 2018 tech talks. This talk, which is entitled “Deep Learning for Real-Time Rendering: Accelerating GPU Inferencing with DirectML and DirectX 12” showcases Nvidia hardware upscaling Playground Games’ Forza Horizon 3 from 1080p to 4K using DirectML in real-time. DirectML has the potential to improve the graphical fidelity of future console and PC games.
We couldn’t write a graphics blog without calling out how DNNs can help improve the visual quality and performance of games. Take a close look at what happens when NVIDIA uses ML to up-sample this photo of a car by 4x. At first the images will look quite similar, but when you zoom in close, you’ll notice that the car on the right has some jagged edges, or aliasing, and the one using ML on the left is crisper. Models can learn to determine the best color for each pixel to benefit small images that are upscaled, or images that are zoomed in on. You may have had the experience when playing a game where objects look great from afar, but when you move close to a wall or hide behind a crate, things start to look a bit blocky or fuzzy – with ML we may see the end of those types of experiences.
DirectML will be a big deal for next-generation graphics
Microsoft sees two ways to push the gaming market forward, utilising both Machine Learning and Ray Tracing technologies to enhance computer-generated graphics to levels that have never been seen before. DirectX Raytracing has arrived on Windows 10 as part of Microsoft’s October 2018 update, and DirectML support came later with Windows 10’s 2019 OS updates.
While DirectML hasn’t received as much attention as DirectX raytracing, you can be sure that developers are looking at the new API closely. As screen manufacturers are starting to push beyond 4K, AI upscaling technologies like Nvidia’s DLSS will continue to increase in popularity. 4K gaming is already a challenge, and 8K is going to prove to be even more problematic for game makers and hardware vendors. Technologies like DirectML will become vital for future games and consoles; the question now is whether or not this tech will be supported on next-generation consoles.
You can join the discussion on Microsoft’s DirectML API on the OC3D Forums.