Comment by jayd16
4 months ago
I don't know, I think it's conceivable that you could get much much better results from a custom upscale per game.
You can give much more input than a single low res frame. You could throw in motion vectors, scene depth, scene normals, unlit color, you could separately upscale opaque, transparent and post process effect... I feel like you could really do a lot more.
Plus, aren't cellphone camera upscalers pretty much realtime these days? I think you're comparing generating an image to what would actually be happening.
> I think it's conceivable that you could get much much better results from a custom upscale per game.
> You can give much more input than a single low res frame. You could throw in motion vectors, scene depth, scene normals, unlit color, you could separately upscale opaque, transparent and post process effect... I feel like you could really do a lot more.
NVIDIA has already been down that road. What you're describing is pretty much DLSS, at various points in its history. To the extent that those techniques were low-hanging fruit for improving upscaler quality, it's already been tried and adopted to the extent that it's practical. At this point, it's more reasonable to assume that there isn't much low-hanging fruit for further quality improvements in upscalers without significant hardware improvements, and that the remaining artifacts and other downsides are hard problems.
Could be true, but not sure how good the results can theoretically become. According to Cerny's words from an interview the current PS5 pro upscaler is tuned specifically for popular playstation games, i.e. they have some extensive testing to catch artefacts.
However it can be a way to ship a smaller and less capable model which fits limited memory and is working just good enough.