[WIP] Module on "Workflow and piping"

workflow_piping/index.qmd

@@ -0,0 +1,220 @@
+---
+title: "Workflow design and piping"
+description: ""
+author: "Etienne Bacher"
+date: "2026-07-27"
+categories: [r]
+difficulty: Beginner
+toc: true
+format:
+  html: default
+  revealjs:
+    output-file: index-slides.html
+execute:
+  warning: false
+  message: false
+  freeze: auto
+editor: 
+  markdown: 
+    wrap: 72
+---
+
+# Piping
+
+## Example where we don't use the pipe
+
+Let's say we want to process the `mtcars` dataset as follows:
+
+* keep the first 10 rows 
+* keep the observations where `cyl >= 6`
+* sort the remaining data by the `am` column.
+
+We could do this in two different ways: by assigning intermediate output or by nesting function calls.
+
+### Intermediate objects
+
+First, we can assign each function output to an object.
+This object can keep the same name and be overwritten at each step:
+
+```{r}
+# Same name gets overwritten
+res <- head(mtcars, 10)
+res <- subset(res, cyl >= 6)
+res <- sort_by(res, ~ am)
+res
+```
+
+However, if we want to rename `res` in the future then we must be careful to rename all its occurrences throughout the code. It also means that if, say, the call to `sort_by()` is wrong, then we must run the entire block again so that `res` is properly reset. Depending on the data size and operations to run, this can be very time-consuming.
+
+An alternative is to use a collection of temporary names:
+```{r}
+# Temporary names
+tmp1 <- head(mtcars, 10)
+tmp2 <- subset(tmp1, cyl >= 6)
+res <- sort_by(tmp2, ~ am)
+res
+```
+
+But this is also not ideal because we pollute the global environment with potentially many temporary objects. Additionally, using a counter in a temporary name means that we need to update many names if we want to add an operation between the first and second step for instance.
+
+
+### Nested calls
+
+The second approach isn't to define intermediate objects, but instead to run all those calls at once by nesting functions:
+
+```{r}
+res <- sort_by(subset(head(mtcars, 10), cyl >= 6), ~ am)
+res
+
+# Same code with different formatting
+res <- sort_by(
+  subset(
+    head(mtcars, 10),
+    cyl >= 6
+  ),
+  ~ am
+)
+res
+```
+
+We didn't define intermediate objects, but to read this code we now need to start from the innermost code (`head(mtcars, 10)`) and expand outwards. 
+This may hurt code readability.
+
+
+## Introducing the pipe
+
+### Using "`|>`"
+
+The pipe is a way to chain operations in natural reading order by automatically passing the output of the left-hand side code to the right-hand side. 
+Using the pipe `|>` with the example above would give:
+
+```{r}
+res <- head(mtcars, 10) |> 
+  subset(cyl >= 6) |> 
+  sort_by(~ am)
+res
+```
+
+The pipe can be read as "and then": we keep the first 10 rows, *and then* we apply our filter, *and then* we sort the remaining data.
+
+:::{.callout-note title="Assigning the output" collapse="true"}
+While we have used `<-` to assign the output above, we could have used `->` to move the assignment to the end of the chain:
+
+```{r}
+head(mtcars, 10) |> 
+  subset(cyl >= 6) |> 
+  sort_by(~ am) -> res
+
+# Equivalent:
+head(mtcars, 10) |> 
+  subset(cyl >= 6) |> 
+  sort_by(~ am) -> 
+  res
+```
+:::
+
+We now have some code that doesn't require intermediate objects but is also easy to read because we see operations in the order in which they are executed.
+
+The `|>` operator was introduced in R 4.1, released in 2021, meaning that you cannot use it in older versions of R.
+
+:::{.callout-note title="Error: The pipe operator requires a function call as RHS" collapse="true"}
+Note that `|>` requires a function call on the right-hand side:
+
+```{r}
+1:3 |> mean()
+```
+```{r}
+#| error: true
+1:3 |> mean
+```
+
+This differs from the `magrittr` pipe, `%>%`.
+:::
+
+
+### Using "`_`"
+
+In the code above, we could seamlessly chain operations because each of those functions take the input data as their first argument.
+
+This is not always the case. For instance, `grepl()` (which detects whether elements of a character vector match a specific pattern) takes the vector to check in second position:
+
+```{r}
+let <- letters[1:6]
+grepl("a|e", let)
+```
+
+Using `|>` without specifying the position of the input would lead to wrong code:
+
+```{r}
+#| warning: true
+let |> 
+  grepl("a|e")
+```
+
+The code above is equivalent to `grepl(let, "a|e")`, which is not what we want and is the cause of the warning. 
+We want to tell `grepl()` that the data we're passing via `|>` should end up in second position.
+To do so, we can use `_`:
+
+```{r}
+let |> 
+  grepl("a|e", x = _)
+```
+
+Some operations don't have named arguments but still require specifying `_`, e.g. to extract a column and then compute its mean:
+
+```{r}
+mtcars |> 
+  _$drat |> 
+  mean()
+```
+
+
+Note that using `_` comes with a few restrictions:
+
+- this operator is available since R 4.2 (released in 2022);
+- `_` must be used on a named argument (except in some cases, such as `$` shown above). For example, this fails:
+  ```{r}
+  #| error: true
+  let |> 
+    grepl("a|e", _)
+  ```
+- a function call can contain only one `_`:
+  ```{r}
+  #| error: true
+  mtcars$drat |> 
+    cor(x = _, y = _)
+  ```
+
+
+### Using anonymous functions in a piped chain
+
+So far, we have only used functions provided in base R (though we could have used functions from other packages).
+Sometimes, it is necessary to run custom code on the data without creating a dedicated new function for that, i.e. we want to use an *anonymous* function.
+If you have used one of the `*apply()` functions before, then you may have used anonymous functions:
+
+```{r eval = FALSE}
+# The function below is anonymous: it isn't assigned to anything, we 
+# just create it on the fly:
+lapply(my_list, function(x) {
+  (x - mean(x) / sd(x))
+})
+```
+
+To use an anonymous function in a piped chain, we need to wrap it in parentheses and evaluate it with `()` at the end of its definition:
+
+```{r}
+mtcars |> 
+  subset(cyl == 4) |> 
+  (function(d) lm(mpg ~ disp, data = d))()
+```
+
+
+### What about "`%>%`"?
+
+You may have seen code that chains operations using `%>%`.
+This `%>%` is also a pipe and it is provided by the `magrittr` package.
+It predates `|>` and was widely used in the [`tidyverse`](https://tidyverse.org/). 
+Its design and popularity inspired the implementation of `|>` in base R in 2021.
+
+For simple cases, `|>` and `%>%` behave identically.
+We recommend using `|>` simply because it is always available in R and doesn't rely on an external package (and the [Tidyverse style guide](https://style.tidyverse.org/pipes.html) also recommends `|>`).