diff --git a/workflow_piping/index.qmd b/workflow_piping/index.qmd new file mode 100644 index 0000000..6b9744b --- /dev/null +++ b/workflow_piping/index.qmd @@ -0,0 +1,220 @@ +--- +title: "Workflow design and piping" +description: "" +author: "Etienne Bacher" +date: "2026-07-27" +categories: [r] +difficulty: Beginner +toc: true +format: + html: default + revealjs: + output-file: index-slides.html +execute: + warning: false + message: false + freeze: auto +editor: + markdown: + wrap: 72 +--- + +# Piping + +## Example where we don't use the pipe + +Let's say we want to process the `mtcars` dataset as follows: + +* keep the first 10 rows +* keep the observations where `cyl >= 6` +* sort the remaining data by the `am` column. + +We could do this in two different ways: by assigning intermediate output or by nesting function calls. + +### Intermediate objects + +First, we can assign each function output to an object. +This object can keep the same name and be overwritten at each step: + +```{r} +# Same name gets overwritten +res <- head(mtcars, 10) +res <- subset(res, cyl >= 6) +res <- sort_by(res, ~ am) +res +``` + +However, if we want to rename `res` in the future then we must be careful to rename all its occurrences throughout the code. It also means that if, say, the call to `sort_by()` is wrong, then we must run the entire block again so that `res` is properly reset. Depending on the data size and operations to run, this can be very time-consuming. + +An alternative is to use a collection of temporary names: +```{r} +# Temporary names +tmp1 <- head(mtcars, 10) +tmp2 <- subset(tmp1, cyl >= 6) +res <- sort_by(tmp2, ~ am) +res +``` + +But this is also not ideal because we pollute the global environment with potentially many temporary objects. Additionally, using a counter in a temporary name means that we need to update many names if we want to add an operation between the first and second step for instance. + + +### Nested calls + +The second approach isn't to define intermediate objects, but instead to run all those calls at once by nesting functions: + +```{r} +res <- sort_by(subset(head(mtcars, 10), cyl >= 6), ~ am) +res + +# Same code with different formatting +res <- sort_by( + subset( + head(mtcars, 10), + cyl >= 6 + ), + ~ am +) +res +``` + +We didn't define intermediate objects, but to read this code we now need to start from the innermost code (`head(mtcars, 10)`) and expand outwards. +This may hurt code readability. + + +## Introducing the pipe + +### Using "`|>`" + +The pipe is a way to chain operations in natural reading order by automatically passing the output of the left-hand side code to the right-hand side. +Using the pipe `|>` with the example above would give: + +```{r} +res <- head(mtcars, 10) |> + subset(cyl >= 6) |> + sort_by(~ am) +res +``` + +The pipe can be read as "and then": we keep the first 10 rows, *and then* we apply our filter, *and then* we sort the remaining data. + +:::{.callout-note title="Assigning the output" collapse="true"} +While we have used `<-` to assign the output above, we could have used `->` to move the assignment to the end of the chain: + +```{r} +head(mtcars, 10) |> + subset(cyl >= 6) |> + sort_by(~ am) -> res + +# Equivalent: +head(mtcars, 10) |> + subset(cyl >= 6) |> + sort_by(~ am) -> + res +``` +::: + +We now have some code that doesn't require intermediate objects but is also easy to read because we see operations in the order in which they are executed. + +The `|>` operator was introduced in R 4.1, released in 2021, meaning that you cannot use it in older versions of R. + +:::{.callout-note title="Error: The pipe operator requires a function call as RHS" collapse="true"} +Note that `|>` requires a function call on the right-hand side: + +```{r} +1:3 |> mean() +``` +```{r} +#| error: true +1:3 |> mean +``` + +This differs from the `magrittr` pipe, `%>%`. +::: + + +### Using "`_`" + +In the code above, we could seamlessly chain operations because each of those functions take the input data as their first argument. + +This is not always the case. For instance, `grepl()` (which detects whether elements of a character vector match a specific pattern) takes the vector to check in second position: + +```{r} +let <- letters[1:6] +grepl("a|e", let) +``` + +Using `|>` without specifying the position of the input would lead to wrong code: + +```{r} +#| warning: true +let |> + grepl("a|e") +``` + +The code above is equivalent to `grepl(let, "a|e")`, which is not what we want and is the cause of the warning. +We want to tell `grepl()` that the data we're passing via `|>` should end up in second position. +To do so, we can use `_`: + +```{r} +let |> + grepl("a|e", x = _) +``` + +Some operations don't have named arguments but still require specifying `_`, e.g. to extract a column and then compute its mean: + +```{r} +mtcars |> + _$drat |> + mean() +``` + + +Note that using `_` comes with a few restrictions: + +- this operator is available since R 4.2 (released in 2022); +- `_` must be used on a named argument (except in some cases, such as `$` shown above). For example, this fails: + ```{r} + #| error: true + let |> + grepl("a|e", _) + ``` +- a function call can contain only one `_`: + ```{r} + #| error: true + mtcars$drat |> + cor(x = _, y = _) + ``` + + +### Using anonymous functions in a piped chain + +So far, we have only used functions provided in base R (though we could have used functions from other packages). +Sometimes, it is necessary to run custom code on the data without creating a dedicated new function for that, i.e. we want to use an *anonymous* function. +If you have used one of the `*apply()` functions before, then you may have used anonymous functions: + +```{r eval = FALSE} +# The function below is anonymous: it isn't assigned to anything, we +# just create it on the fly: +lapply(my_list, function(x) { + (x - mean(x) / sd(x)) +}) +``` + +To use an anonymous function in a piped chain, we need to wrap it in parentheses and evaluate it with `()` at the end of its definition: + +```{r} +mtcars |> + subset(cyl == 4) |> + (function(d) lm(mpg ~ disp, data = d))() +``` + + +### What about "`%>%`"? + +You may have seen code that chains operations using `%>%`. +This `%>%` is also a pipe and it is provided by the `magrittr` package. +It predates `|>` and was widely used in the [`tidyverse`](https://tidyverse.org/). +Its design and popularity inspired the implementation of `|>` in base R in 2021. + +For simple cases, `|>` and `%>%` behave identically. +We recommend using `|>` simply because it is always available in R and doesn't rely on an external package (and the [Tidyverse style guide](https://style.tidyverse.org/pipes.html) also recommends `|>`). \ No newline at end of file