integrator-using functions should return NaN on non-successful integrator steps

[Funkerfish]

Hello again!

Now I try to calculate the Lyapunov spectrum in a two-parameter plane. But a warning comes when I choose some of the parameter regions:

Warning: Instability detected. Aborting

From what I googled, the warning comes because it returns a NAN in the trajectory. I am thinking if I can use the 'isnan' function to skip those parameter sets that cause the NAN and continue my loop. Since the lyapunovspectrum function is built-in, I am looking for your help.

Here is my code:

using DynamicalSystems, CairoMakie, GLMakie
using OrdinaryDiffEq
using DelimitedFiles
using ProgressMeter
function chaotictest(u, p, t)
    k1, k2 = p
    x, y, z = u
    dx = y
    dy = z
    dz = k1-7*y+x^2+k2*z
    return SVector(dx, dy, dz)
end
u0 = [0.01, 0.01, 0.01]
para = [-1.0, -1.0] 
ds = ContinuousDynamicalSystem(chaotictest, u0, para)
diffeq = (alg = Tsit5(), adaptive = false, dt = 0.001)
as = -1:0.1:8;  
bs = -1.0:0.05:0.01;
nloop = length(as)*length(bs)
cordiλ = zeros(nloop, 2+3)
λs = zeros(nloop, 3)
paras = zeros(nloop, 2)
cirnum = 0
prog = Progress(nloop; showspeed=true)
for (i, a) in enumerate(as)
    for (j, b) in enumerate(bs)
         global cirnum = cirnum + 1
    set_parameter!(ds, 1, a)
    set_parameter!(ds, 2, b)  
    λs[cirnum, :] .= lyapunovspectrum(ds, 2000; Ttr=3000, diffeq)
    paras[cirnum, :] .= [a, b]
    global cordiλ = [paras λs]
    next!(prog)
end
end

[Funkerfish]

when the diction of parameters is set as:

as = 2:0.1:8;  
bs = -0.9:0.05:0.01;

the warning comes immediately.

[Datseris]

what does lyapunovspectrum return for these cases that there is an instability...?

[Funkerfish]

it seems that the loop would never end when the instability warning comes, I don't know but I guess it would be the NaN, so as the function trajectory() returns.

please check the following codes with the warning：

using DynamicalSystems, CairoMakie, GLMakie
using OrdinaryDiffEq
function chaotictest(u, p, t)
    k1, k2 = p
    x, y, z = u
    dx = y
    dy = z
    dz = k1-7*y+x^2+k2*z
    return SVector(dx, dy, dz)
end
u0 = [0.01, 0.01, 0.01]
para = [2, -0.9] #k2=k6=0
ds = ContinuousDynamicalSystem(chaotictest, u0, para)
diffeq = (alg = Tsit5(), adaptive = false, dt = 0.001)
test111 = trajectory(ds, 20; Ttr=0, diffeq)

I think the warning may be solved by changing the solver. But since it works in some of the parameter sets, we can't do that during a loop, and that is why I need your help.

[Datseris]

it seems that the loop would never end when the instability warning comes, I don't know but I guess it would be the NaN

The answer to my question is not clear though. What is lyapunovspectrum doing if you give it an initial condition you know will give you "instability detected, aborting"? Is the function never returning? is it returning NaNs? Is it returning something else?

[Funkerfish]

lyapunovspectrum never returns once the warning comes. I check the matrix which saves the values and find that it still be zero (already pre-assign 0 values to the matrix)

[Datseris]

Okay. So we need to make lyapunovspectrum terminate when the integrator detects instability. What do you think the lyapunovspectrum function should return when this happens?

[Datseris]

If you want to do a PR that solves this, check for the success field in the integrator sturcture, which becomes false if the step wasn't successful

[Funkerfish]

Because this problem comes from the ODE solver, the same warning would take place in other functions like Interactive Trajectory Evolution when we slide the bar of parameters.

I think these functions should return 'NaN' value so that we can create a filter by applying the isnan function. In old times (when using Fortran), we would let ODE solver do calculations for few steps and check if there are 'Nan' values. If so, we can skip to the next parameter set.

Thank you for your advice, I would try success field. Your work is awesome!

[Datseris]

Because this problem comes from the ODE solver, the same warning would take place in other functions like Interactive Trajectory Evolution when we slide the bar of parameters.

Yes, that is true, but the only way to handle this, as far as I can tell, is to have such a special handling of the integrator for each individual function that uses an integrator. I don't see any other way.

I think these functions should return 'NaN' value so that we can create a filter by applying the isnan function

What's the benefit to that? If the integrator structure already has a success field, then doesn't this already give you the info you need? What is the additional benefit of making the state NaN as well as having a false return code for success?

Your work is awesome!

You are very welcome!

[Funkerfish]

Of course, the success field would work. But my point is, many people like me would use more than one coding language to do the calculation, for example, obtaining the timeseries in Julia and analyzing them in Matlab, so it is important that the structure of data matrix keeps consistent. As far as I know, Functions like ODE45 in Matlab would return 'NaN' values and it would be great if DynamicalSystems.jl do the same.

Besides, the instability thing may not happen if we choose another solver, of course It is our duty to choose the proper one but it would be great to leave a ‘NaN’ mark in the data matrix to help us to evaluate them.

[Datseris]

Okay, I didn't understand you in the first place. You were suggesting that functions like lyapunovspectrum should return NaN in the case of instability, which I completely agree with. What I originally thought is that you were suggesting that the integrator should make the state NaN, which I don't agree with. Of course, now someone has to actually do the pull requests in functions like lyapunovspectrum and make then return NaN in case of instability.

But my point is, many people like me would use more than one coding language to do the calculation, for example, obtaining the timeseries in Julia and analyzing them in Matlab, so it is important that the structure of data matrix keeps consistent.

I need to tell you that for me this is not an argument. Thankfully I agree with you for different reason, that NaN is a sensible design just by itself. But saying "MatLab does it this way" is not convincing for me, not only because I consider Julia vastly superior, but also because by itself "someone else does it different" isn't necessarily a logically convincing argument.

But I am curious. Why are you doing your timeseries analysis in Matlab instead of Julia...? What tools are missing in Julia that you have in Matlab?

[Funkerfish]

What is the additional benefit of making the state NaN as well as having a false return code for success?

To be more specific, please allow me to show you an example:
Now we have a 5x2 Matrix to store the parameters and the corresponding Lyapunov exponent (pre-assigned to 0),

parameters      Lyapunov exponents
  0.0                 0.0
  0.0                 0.0
  0.0                 0.0
  0.0                 0.0
  0.0                 0.0

The parameter set is 1 to 5 and we assume the instability comes when taking the parameter at 3 and 4. Let's see what will happen with NaN compared to false code:

parameters      Lyapunov exponents 
  1.0                 -1.2   
  2.0                 -1.1           
  3.0                  NaN               
  4.0                  NaN             
  5.0                 -0.9  

parameters      Lyapunov exponents 
  1.0                 -1.2   
  2.0                 -1.1           
  5.0                 -0.9              
  0.0                  0.0           
  0.0                  0.0  

Obviously, false will destroy the original data structure. What if the function is not lyapunovspectrum but one defined by ourselves? for example, we define one function to calculate the distance between two trajectories:

dd = x(t) - y(t)

How can we let them return NaN values? So, the final solution is to let the integrator make the state NaN.

[Funkerfish]

But I am curious. Why are you doing your timeseries analysis in Matlab instead of Julia...?

I really agree with you that Julia is superfast when doing calculation and that is the reason I turn to this language and DynamicalSystems.jl.

Just imagine that we got a 300x3 data matrix in Julia and you want to extract a few lines of values as your wish or just sort values by one row. I think it would be more convenient to open the data file in Matlab or even in Microsoft Excel. But we can't say Microsoft Excel is better than Julia, can we?

I hold the opinion that in some ways DynamicalSystems.jl should act like other classical languages so that it could be more friendly and attractive to those starters.

[Datseris]

Let's see what will happen with NaN compared to false code:

That's not what I am suggesting though... Integrator returns false on instability, but lyapunovspectrum returns NaN by detecting the instability of the integrator. In any case, since we agree on the outcome there's no reason to discuss this further. lyapunovspectrum should return NaN on trajectory instability.

Now someone has to actually do the PR that implements this.

Just imagine that we got a 300x3 data matrix in Julia and you want to extract a few lines of values as your wish or just sort values by one row. I think it would be more convenient to open the data file in Matlab or even in Microsoft Excel.

This doesn't make sense to me, why not just use the built in sort! function of Julia...? you'd literally call sort(matrix; dims = 2). From my perspective this is much more convenient than opening Excel :D

[Funkerfish]

Okey, I will look forward to this update! Would we still see the warning message in REPL before returning NaN? because its quite annoying and would block the progress bar.

Thanks for all your warm replies!