Hi
I am working on a problem that requires learning how to select the some elements of x such that the downstream task that explicitly depends on x is optimized.
The problem that I am facing is P@x gives elements which are very different from x. I understand that there is some-smoothing going on to make the sorting differentiable. But is it expected that the soft-sort would drastically change the elements values. I notice that increasing the steepness factor minimizes the changes in x. But then, the training is very unstable and the loss does not decrease.
How should I tune the parameters of diffsort?
Arun
Hi
I am working on a problem that requires learning how to select the some elements of x such that the downstream task that explicitly depends on x is optimized.
The problem that I am facing is P@x gives elements which are very different from x. I understand that there is some-smoothing going on to make the sorting differentiable. But is it expected that the soft-sort would drastically change the elements values. I notice that increasing the steepness factor minimizes the changes in x. But then, the training is very unstable and the loss does not decrease.
How should I tune the parameters of diffsort?
Arun