Nvidia promises a brighter future for DLSS – New techniques will deliver greater detail

    One of the core challenges of super resolution is preserving details in the image while also maintaining temporal stability from frame to frame. The sharper an image, the more likely you’ll see noise, shimmering, or temporal artifacts in motion.

During our research, we found that certain temporal artifacts can be used to infer details in an image. Imagine, an artifact we’d normally classify as a “bug,” actually being used to fill in lost image details. With this insight, we started working on a new AI research model that used these artifacts to recreate details that would otherwise be lost from the final frame.

This AI research model has made tremendous progress and produces very high image quality. However, we have work to do to optimize the model’s performance before bringing it to a shipping game.

Leveraging this AI research, we developed a new image processing algorithm that approximated our AI research model and fit within our performance budget. This image processing approach to DLSS is integrated into Control, and it delivers up to 75% faster frame rates.

Let’s look at an example video, below. The left side uses Control’s in-engine scaling. The right side shows DLSS. Both sides are rendering at 720p, and outputting at 1080p. Notice how DLSS brings out more detail and improves temporal stability, reducing flickering and shimmering.

While the image processing algorithm is a good solution for Control, the approximation falls short in handling certain types of motion. Let’s look at an example of native 1080p vs. 1080p DLSS in Control. Notice how the flames on the right are not as well defined as in native resolution.

Clearly, there’s opportunity for further advancement.

Nvidia knows that DLSS offers levels of image quality that are nowhere near the levels offered by native resolution rendering. More often than not, we have found that the technique falls short when rendering certain types of images. Knowing this, Nvidia is exploring new techniques which will allow DLSS to continue to deliver increased game performance while smoothing over the shortcomings of the technology. 

The video below showcases a new AI research model from Nvidia, which uses new techniques which Nvidia plans to optimise and bring to future games. 

 

    There are many other examples of how deep learning is used to create super resolution images and video, create new frames of video, or transfer an artist’s style from one image to the next. Before Turing, none of this was possible in real-time. With Turing’s Tensor Cores, 110 teraflops of dedicated horsepower can be applied for real-time deep learning.

Let’s look at an example of our image processing algorithm vs. our AI research model. The video below shows a cropped Unreal Engine 4 scene of a forest fire with moving flames and embers. Notice how the image processing algorithm blurs the movement of flickering flames and discards most flying embers. In contrast, you’ll notice that our AI research model captures the fine details of these moving objects.

With further optimization, we believe AI will clean up the remaining artifacts in the image processing algorithm while keeping FPS high.

More Innovation To Come

The increasing computing demands of next generation, ray-traced content requires clever approaches, such as super resolution, to deliver great frame rates. The new DLSS techniques available in Control are our best yet. We’re also continuing to invest heavily in AI super resolution to deliver the next level of image quality.

Our next step is optimizing our AI research model to run at higher FPS. Turing’s 110 Tensor teraflops are ready and waiting for this next round of innovation. When it arrives, we’ll deploy the latest enhancements to gamers via our Game Ready Drivers.

 

Right now, it is clear that Nvidia sees a future in its DLSS technology, hoping that it will become a major differentiating factor between Geforce gaming products and competing offerings from the likes of AMD and Intel moving forward. Hopefully, in time, DLSS will become the image/performance-enhancing feature that was originally promised to gamers at Turing’s launch. 

You can join the discussion on Nvidia’s plans for DLSS on the OC3D Forums.