FIMD2 :: section 5.1 :: Workflow 7 dplyr::do() is superseded

[skolenik]

@stefvanbuuren the documentation for dplyr::do states that it is superseded: https://dplyr.tidyverse.org/reference/do.html.

The code for workflow 7 with the newer versions of dplyr might run as:

library(dplyr)
est7_newer <- nhanes %>%
  mice(seed = 123, print = FALSE) %>%
  mice::complete("long") %>%
  nest_by(.imp) %>%
  mutate(model = list(lm(formula = chl ~ age + bmi + hyp, data = data))) %>%
  as.list() %>%
  purrr::pluck("model") %>%
  pool()

[stefvanbuuren]

Stas, thanks for your note.

Frankly, I was never quite happy with my workflow 7 because it was rather artificial. I also observed that it had stopped working at some point because of the deprecated do().

Thanks for your version of workflow 7. It feels a bit like magic, with these abstract nest_by() and pluck() functions, but it may be useful in some contexts.

[skolenik]

I think with() is magic if you ask me -- I am more comfortable with the tidy workflows as they return predictable tibbles, and I never have a clue what's going to come out of *apply() or with() -- a list? a matrix? a data frame?..

purrr::pluck() is your biggest friend ever -- it is a universal extractor from lists and vectors and data frames by either position or name, so you don't have to agonize figuring out if you need [[number or $name or [name extractor. nest_by() prepares things better for the purrr kinds of workflows, returning a list (in this case, of tibbles).

Interesting that it stopped working for you. This is the only workflow that works for me in some contexts (like subsetting the data before estimation).

[stefvanbuuren]

Thanks for your thoughts. I remain amazed about the many ways to do the same thing in R. It keeps us learning.

[skolenik]

Starting a new subthread here @stefvanbuuren --

I was bothered by infinitely long compute times of some multinomial logistic regressions I had to run for $M=50$, so I came up with a version of the workflow that uses furrr::future_pmap():

library(furrr)
est7_newest <- nhanes %>%
    mice(seed = 123, print = FALSE) %>%
    mice::complete("long") %>% # only the imputed data, no original data 
    nest_by(.imp) %>%     # replacement for `dplyr::do()`
    future_pmap(.f = function(...) {
        the_dots <- rlang::list2(...)  # this is some magic words
        lm(formula = chl ~ age + bmi + hyp, 
          data = the_dots$data) # this is where the action is but you have to keep the_dots$data language
          # you can also chain data subsetting e.g. data = the_dots$data %>% filter(bmi >= 20)
    }) %>% # this returns a list so it is ready to be pooled
    pool()
testthat::expect_equal(est7_newer, est7_newest)

The user needs to be at least somewhat comfortable with purrr and we can probably say rlang::list2(...) is just voodoo for the average social science user... but then every workflow has a voodoo component to it ;). And this workflow eliminates the need to convert data frames to lists and lift the $model component out of it -- that was the voodoo of The FIMD Workflow 7.

[skolenik]

I will use these both as examples in my FARS vignette, I am not ready to formally PR it yet, but I am making progress https://github.com/skolenik/mice/blob/fars-vignette/vignettes/FARS-mice.Rmd

[thomvolker]

@skolenik I like the approach! You might consider the following, which, to me, seems to cut out some of the voodoo, and only requires people being familiar with map-like functionality and the tidyverse.

nhanes %>%
  mice(seed = 123, print = FALSE) %>%
  mice::complete("long") %>% # only the imputed data, no original data 
  nest_by(.imp) %>%
  pluck("data") %>%
  future_map(~lm(chl ~ age + bmi + hyp, .x)) %>%
  pool()

Or like so (but I like the previous better because you can do data manipulation with the .x variable):

nhanes %>%
  mice(seed = 123, print = FALSE) %>%
  mice::complete("long") %>% # only the imputed data, no original data 
  nest_by(.imp) %>%
  pluck("data") %>%
  future_map(lm, formula = chl ~ age + bmi + hyp) %>%
  pool()