How to do multiple progress bars, especially when running nested futures?

[angel-bee2018]

Hi,

For nested future.apply::future_apply, furrr::future_map and purrr::map, is it possible to construct a progress report neatly so that the hierarchical arrangement of sub-loops or child processes is clear. e.g. something like this (kind of like str):

plan(list(tweak(multicore, workers = 2),
              tweak(multicore, workers = 3)
       ) )

future.apply::future_apply(
    X = tibble("a" = 100:200),
    MARGIN = 1,
    FUN = function(a1) {

        future.apply::future_apply(
        X = tibble("b" = 1:a1),
        MARGIN = 1,
        FUN = function(b1) {

            return(b1^2)

        } )

    } )

Level 1 #1 |====================== | 64%
.... Level 2 #1 |=============================== | 90%
.... Level 2 #2 |====================== | 65%
.... Level 2 #3 |============ | 37%
Level 1 #2 |==================== | 43%
.... Level 2 #1 |==================== | 43%
.... Level 2 #2 |==================== | 43%
.... Level 2 #3 |==================== | 43%

From the vignette, it doesn't seem apparent as to how this may be done - it only contains examples to show only one overall bar. Is this a thing that can be done with progressr?

thanks in advance

[HenrikBengtsson]

Hi.

However neat that would be, unfortunately, nested progress updates are not supported. More specifically, the progress handlers can only listen to updates from the top layer.

Having said that, a first baby step would be to support "still-alive" updates from progress signaled from nested layers, cf. #107. That's doable, because such info will correspond to sending p(amount = 0) at the top level, which can be rendered as a "spinner" in the terminal.

When it comes to rendering multi-line progress info in the terminal, that's poorly supported across platforms and environments, e.g. I don't think the RStudio Console can do that. This is why most progress frameworks in R are limited to single-line presentations. This type of feature is unlikely something that will be resolved in this package (progressr). That's more something for the packages and functions that renders progress bars, e.g. progress and cli. If they can solve it in a platform & environment agnostic way, then we might be able redesign progressr to support them.

Now, a non-multiline alternative, could be a multi-part, single-line progress bar, e.g.

48% |==   |==   |====|=== |==   |=    |

and

48%▕░▓▓░░▓░░▓░░░▓▓▓▓░▓▓▓▏

But, the objective is not to implement that in progressr, but rely on other packages to handle those. The progressr package should just provide them with the proper information.