The Nvidia AI interview: Inside DLSS 4 and machine learning with Bryan Catanzaro
At CES 2025, Nvidia announced its RTX 50-series graphics cards with DLSS 4. While at the show, we spoke with Nvidia VP of applied deep learning research Bryan Catanzaro about the finer details of how the new DLSS works, from its revised transformer model for super resolution and ray reconstruction to the new multi frame generation (MFG) feature.
Despite coming just over a year since our last interview with Bryan, which coincided with the release of DLSS 3.5 and Cyberpunk 2077 Phantom Liberty, there are some fairly major advancements here, some of which that will be reserved for RTX 50-series owners and others that will be available for a wider range of Nvidia graphics cards.
The interview follows below, with light edits for length and clarity as usual. The full interview is available via the video embed below if you prefer. Enjoy!
00:00 Introduction00:48 Why switch from CNNs to transformers?02:08 What are some image characteristics that are improved with DLSS 4 Super Resolution?03:17 Is there headroom to continue to improve on Super Resolution?04:12 How much more expensive is DLSS 4 Super Resolution to run?05:25 How does the transformer model improve Ray Reconstruction?09:43 Why is frame gen no longer using hardware optical flow?13:06 Could the new Frame Generation run on RTX 3000?13:44 What has changed for frame pacing with DLSS 4 Frame Generation?15:37 Will Frame Generation ever support standard v-sync?17:18 Could you explain how Reflex 2 works?21:11 What is the lowest acceptable input frame-rate for DLSS 4 Frame Generation?22:13 What does the future of real-time graphics look like?
The last time we talked was when ray reconstruction first came out, and now, with RTX 5000, there’s a new DLSS model – the first time since 2020 that we’re seeing such a big change in how things are done. So why switch over to this new transformer model? To start, how does it improve super resolution specifically?