Caustic Rendering

[CycloneRing]

Hi, Is there any way to enable caustics?

[CycloneRing]

It seems basic caustics are supported.

[fstrugar-nv]

Hey sorry it took me ages to answer :) I'll reopen until we actually have solution for real-time caustics - that's one of the "holy grails".

You'll get caustics in reference mode if you're patient enough (accumulate very many frames) and disable firefly filter. But since RTX PT is currently just a basic unidirectional path tracer, there's no way for it to figure out complex paths that are "good" for connecting to lights through multiple refractions.

What we have now is good NEE light sampling - but that only gives us direct illumination at each bounce, which caustics are not - they require finding lights across a chain of bounces.

In the future we'll (hopefully) have two options that effectively "learn" how to find these good paths: ReSTIR PT, and some form of path guiding.

[CycloneRing]

@fstrugar-nv Thank you for response. I played around the demo and I can see caustics in DLSS RR. Is this happening by accident?

One issue is we don't get transparent shadows if shadow be casted from a transmitted material... Any hope to get this fixed?

[fstrugar-nv]

It's not by accident (well ok everything's by accident in a way😆) - if you leave it for long enough to accumulate enough samples, it'll converge to the ground truth eventually.

In the above example, the main path will bounce off the ground and hit the sphere, refract into the sphere (if it doesn't reflect off), maybe bounce few times inside and then refract out and if you're lucky hit a light source. In most cases the first diffuse bounce off the ground won't hit the sphere at all as it is not guided in any way.

Since RR is very good at reconstructing signal from little data, in some simpler cases like this it will actually catch the output of these rare paths (which look like noisy fireflies in the input signal), and reconstruct some of the caustics. But in case when these are very rare then -RR will also produce unstable, boiling output. This is why there's the "Firefly Filter" option - it clamps noise that's "too noisy".
If in the Realtime mode you increase the number of samples per pixel above 1 and reduce/disable FireflyFilter, you'll get a lot better caustics. But that's not very practical from performance standpoint.

One issue is we don't get transparent shadows if shadow be casted from a transmitted material... Any hope to get this fixed?

some of these shadows should not be transparent as in a lot of cases it's just the photons not reaching the ground at all due to reflection or refraction - they're not caused by the volumetric/surface absorption of the material at all!

some of the darkening is because of firefly filter and lack of signal.

best way to distinguish between the 2 is to capture a very many sample reference with firefly filter disabled!

here's a 40k reference:

here's a 1spp Realtime mode (default settings):

here's a 8spp Realtime mode with firefly filter disabled:

The idea is to get the 8spp quality at the cost of not much more than 1spp and that should be possible with path guiding or ReSTIR PT - that's in the future :)

[CycloneRing]

@fstrugar-nv Thank you for great explanation.

some of these shadows should not be transparent as in a lot of cases it's just the photons not reaching the ground at all due to reflection or refraction - they're not caused by the volumetric/surface absorption of the material at all!

What I meant is if we have a mesh with shell which is used to simulate tinted glass in reference mode we get transparent shadows

But in real-time we have no such thing

For caustics can't it be done similar to the approach was done in Falcor and then ported to UE?
https://github.com/league1991/RTCaustics
https://github.com/NvRTX/UnrealEngine/tree/NvRTX_Caustics-4.26
https://developer.nvidia.com/blog/generating-ray-traced-caustic-effects-in-unreal-engine-4-part-1/

I also noticed point and spot lights don't work and LightBaker only bakes directional lights. In light information tab I can see Spot and Directional counted as analytic but no point light is counted. I debugged the code and I see the lights are imported from GLTF by GLTF Importer but no light will be emitted in scene from point and spot lights regarding Read Me pointing they are supported, Am I missing something or this is an issue?

[fstrugar-nv]

What I meant is if we have a mesh with shell which is used to simulate tinted glass in reference mode we get transparent shadows
But in real-time we have no such thing

As I've noted in previous post, if you increase the number of samples, and disable firefly filter, you'll get tinted shadows in realtime mode as well. But you'll never get a good signal that way, because effectively you're getting no lighting from NEE (because that works only for opaque surfaces) so you only have BSDF sampling and no light importance sampling, and that's not good enough for diffuse surfaces.

https://pbr-book.org/3ed-2018/Light_Transport_I_Surface_Reflection/Direct_Lighting figure 14.13 shows how BSDF and light (NEE) sampling work together. But in your case, the shadowed areas get no benefit from light sampling because that can only correctly evaluate direct lighting, and you're left with BSDF sampling which is too noisy.

So, you either need a way to guide your BSDF (path) samples towards the light (path guiding or ReSTIR PT), or some other technique.

You CAN also cheat by ignoring refraction and enabling light sampling (NEE) for (some) transparencies, with support for tinted shadows. I'll implement that at some point as it seems like an easy (although physically incorrect) solution for a lot of these cases.

For caustics can't it be done similar to the approach was done in Falcor and then ported to UE?

Those examples use photon mapping - that's another way to do it, but I don't think it's practical for large scenes and multiple generic light sources. It'd be nice to implement though.

I also noticed point and spot lights don't work and LightBaker only bakes directional lights. In light information tab I can see Spot and Directional counted as analytic but no point light is counted. I debugged the code and I see the lights are imported from GLTF by GLTF Importer but no light will be emitted in scene from point and spot lights regarding Read Me pointing they are supported, Am I missing something or this is an issue?

Point lights should currently be handled as tiny spherical lights, so there's probably a bug somewhere if they don't work :( This is one of the things that I need to revisit at some point.

[CycloneRing]

@fstrugar-nv Thank you for the time, Yeah all makes sense, I'll take a look and I see if I can fix the point light issue. How spot lights are handled? Do they work at the moment? I have no emission from them while they are presented.

I'll implement that at some point as it seems like an easy (although physically incorrect) solution for a lot of these cases.

That would be great.
I'm very optimistic about this project. Thank you for all your efforts ❤

[fstrugar-nv]

Thank you! I've just added it to the list of tasks. I don't know when I'll get around to do it, but it shouldn't be too complicated.

I'll leave the issue open as a reminder!