diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 0e2e290..cb37007 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -48,6 +48,11 @@ jobs: with: use-public-rspm: true + - name: Sync the Lua filter into each module + # Mirrors _quarto.yml's pre-render hook that runs locally. + working-directory: modules + run: Rscript sync_filter.r + - name: Copy PR module(s) into the website shell: Rscript {0} env: diff --git a/.gitignore b/.gitignore index 9c6e3c2..1a4c12f 100644 --- a/.gitignore +++ b/.gitignore @@ -9,4 +9,10 @@ _site _freeze .DS_Store +*_files/ +*.html + **/*.quarto_ipynb + +/.luarc.json +web_and_slides_autogenerated.lua diff --git a/README.md b/README.md index d8f5ced..2c1e1e8 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,190 @@ -List of modules built by the Palaeoverse team. +# Palaeoverse Modules -One folder per module. +List of teaching modules built by the Palaeoverse team. + +One folder per module: `/index.qmd`, plus whatever images and data it +needs. Every module is rendered twice from that single source: 1) as a page on the +[Palaeoverse website](https://palaeoverse.org/training/modules) and 2) as a +reveal.js slide deck (`index-slides.html`) for teaching the same material live. + +The two files at the root of this repo exist to make one source serve both +outputs: + +| File | Role | +| --- | --- | +| `web_and_slides.lua` | Pandoc/Quarto filter, applied at render time. Decides what appears on the website, what appears on the slides, and how the slides are broken up. | +| `web_and_slides.r` | One-shot authoring helper. Converts a finished long-form document into the tagged form the filter expects. | + +# Authoring a module + +## Front matter + +Start from an existing module (`ggplot/index.qmd` is a good template). These +fields matter: + +```yaml +--- +title: "Plotting in R with ggplot2" +description: "Visualizing your data with the grammar of graphics" +author: "Will Gearty" +date: "2026-06-12" +categories: [r, tidyverse, dataviz] # various tags +difficulty: Beginner # Beginner/Intermediate/Advanced +image: images/2d_density.png # thumbnail for module +format: + html: default + revealjs: + smaller: true + output-file: index-slides.html # the deck, alongside the page +execute: + output-location: fragment # slide output reveals on click + echo: true + freeze: auto # must be auto, not true +filters: + - at: pre-ast + path: web_and_slides_autogenerated.lua +--- +``` + +`difficulty`, `categories` and `freeze: auto` are enforced by CI (see +[Technical summary](#technical-summary)). The `filters` entry is what activates +everything below; `web_and_slides.r` adds it for you, or you can copy it. + +## Writing for two outputs + +Three fenced-div (`:::`) classes control where content lands: + +| Class | Website Tutorial | Slides | +| --- | --- | --- | +| `.narration` | normal prose | speaker notes | +| `.slides-only` | dropped | shown on the slide | +| `.html-only` | shown | dropped | + +Anything not wrapped in one of these appears in both outputs. So the usual shape +of a module is: headings and code chunks shared by both outputs, the connecting +prose in `.narration` (a paragraph on the page, a note you talk from on the +slide), and the occasional `.slides-only` bullet summary or `.html-only` aside. + +````markdown +## Making a scatter plot + +::: {.narration} +On the website this is a paragraph of explanation. On the slides it is what you +say out loud while the plot is up. + +Consecutive paragraphs can share one block. +::: + +::: {.slides-only} +- x is body mass +- y is flipper length +::: + +```{r} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len) + + geom_point() +``` +```` + +## Automated slide rendering via `web_and_slides.lua` + +The following changes are applied to the reveal.js slides via our custom Lua filter: + +- **Headings become slides:** `##` starts a slide as usual; `###` and deeper are + promoted so each also gets its own slide, instead of piling onto the parent. + A `#` heading becomes a centered divider slide, so don't put content under one. +- **One plot per slide:** The slide closes after each figure. Prose that follows + a figure moves to the next slide (introducing it), unless nothing but prose + remains before the next heading — then it stays put rather than making a blank + slide. Split slides repeat the current heading, so they keep a title. +- **Callouts get their own slide:** A callout is un-boxed onto a slide of its + own, titled by its own heading. Give a callout a `## Heading` as its first line + (rather than `title="..."`) if you want that title on the slide; an untitled + callout keeps the section title. +- **Multi-chunk slides build up:** A slide holding two or more code chunks is + expanded into an auto-animate sequence: one step per chunk, earlier chunks + staying on screen, and the notes for each chunk advancing with it. + +Preview both outputs with: + +```sh +quarto render ggplot/index.qmd # -> index.html and index-slides.html +``` + +## `web_and_slides.r` Helper + +The `web_and_slides.r` helper file takes a long-form document and mechanically prepares it: + +```sh +Rscript web_and_slides.r [output.qmd] + [--filter=] [--no-inject] [--no-settings] +``` + +It does the following: + +1. Wraps each set of consecutive prose paragraphs in a single + `::: {.narration}` block. Headings, code chunks, lists, blockquotes, + tables, standalone images, and existing fenced divs (including callouts and + everything inside them) are left untouched. +2. Registers the filter in the front matter at the `pre-ast` stage, replacing any + earlier registration. `--no-inject` skips this; `--filter=` names a different + file. +3. Sets slide-friendly YAML (skip with `--no-settings`): + - adds `execute.echo: true` (renders source code on slides) + - adds `execute.output-location: fragment` (renders code results as separate chunk) + - adds `format.revealjs.smaller: true` (text shrinks to fit on slides) + - removes `format.revealjs.scrollable` (disables scrolling through slides) +4. Reports headings that will render awkwardly (e.g., `#` become centered title slides). + You should fix these by hand. + +The front matter is checked for valid YAML before anything is written. Needs the +`readr`, `stringr`, `yaml`, and `fs` packages. + +With no `output.qmd` specified it rewrites the input in place. + +# Two ways to build a module + +## 1. Long-form first, then convert + +_Best when the module is primarily a written tutorial, or already exists as prose._ + +Write the module as an ordinary prose-and-code document, ignoring slides +entirely. When it reads well, run `web_and_slides.r` over it once, then clean up +the result by hand: act on any headings it flagged and add `:::{.slides-only}` / +`:::{.html-only}` blocks where the two outputs should diverge. + +```sh +# write the preliminary tutorial in mymodule/index.qmd.orig +Rscript web_and_slides.r mymodule/index.qmd.orig mymodule/index.qmd +quarto render mymodule/index.qmd +``` + +The script is intended to be run as a **single pass once the long-form version is +finished**. However, in principle it can be run on the same file multiple times. +Prose already inside a `.narration` block is left alone, so a second run over its +own output changes nothing. If you start hand-editing the converted file, keep +editing that file. If you would rather keep iterating on the long-form text, +keep it alongside as `index.qmd.orig` and regenerate `index.qmd` (but any hand +cleanup you did to `index.qmd` will need redoing). + +## 2. Tag as you go + +_Best when you are thinking about the slides and the prose tutorial at the same +time, and when you want fine control over which output gets what._ + +Skip `web_and_slides.r` completely. Write `index.qmd` with the intent to render +both output types from the beginning. Copy the `filters` entry and the +slide-friendly `execute` / `revealjs` settings from another module's front matter. +Then, as you write use an `.html-only` block if you only want the text to +appear in the prose tutorial, a `.slides-only` block if a section should only +appear on the slides (e.g., bullet points), or a `.narration` block have +content appear in the prose tutorial and be shown as speaker notes for the slides. + +# Additional references: + +- Quarto tutorial ([with Positron](https://quarto.org/docs/get-started/hello/positron.html) | [with RStudio](https://quarto.org/docs/get-started/computations/rstudio.html)) +- [Markdown basics in Quarto](https://quarto.org/docs/authoring/markdown-basics.html) +- [HTML basics in Quarto](https://quarto.org/docs/output-formats/html-basics.html) +- [Revealjs in Quarto](https://quarto.org/docs/presentations/revealjs/) +- [Quarto Listings](https://quarto.org/docs/websites/website-listings.html) diff --git a/_quarto.yml b/_quarto.yml new file mode 100644 index 0000000..e9154f6 --- /dev/null +++ b/_quarto.yml @@ -0,0 +1,7 @@ +project: + type: default + # modules only, so README.md / LICENSE.md aren't rendered + render: + - "*/*.qmd" + # refresh each module's copy of the Lua filter before rendering + pre-render: sync_filter.r diff --git a/ggplot/index.qmd b/ggplot/index.qmd index 68c72c7..a0136bf 100644 --- a/ggplot/index.qmd +++ b/ggplot/index.qmd @@ -9,52 +9,69 @@ image: images/2d_density.png format: html: default revealjs: + smaller: true output-file: index-slides.html execute: + output-location: fragment + echo: true warning: false message: false freeze: auto editor: markdown: wrap: 72 +filters: + - at: pre-ast + path: web_and_slides_autogenerated.lua --- +## Introduction -# Introduction - +::: {.narration} While you can make plots with just the packages that come bundled with base R, many R users make their visualizations entirely using the [`ggplot2`](https://ggplot2.tidyverse.org/index.html) package and an [ecosystem of packages](https://exts.ggplot2.tidyverse.org/gallery/) designed around it. +::: ```{r ggplot2} # load the ggplot2 package library(ggplot2) ``` +::: {.narration} As with the previous sessions, we'll be using the Palmer penguins dataset. While we built our own combined dataset in the introduction session, now we're going to use the built-in cleaned dataset. First, let's load the dataset. Then let's inspect the data using the `glimpse()` function. +::: ```{r palmerpenguins} data(penguins) dplyr::glimpse(penguins) ``` +::: {.narration} As we discovered before, this dataset includes many different measurements for individual penguins from three different studies. The studies cover both sexes of three different species of penguins from three different islands in the Palmer Archipelago. +::: -# The ggplot2 basics +## The ggplot2 basics +::: {.narration} The most important function in the `ggplot2` package is `ggplot()`. Note that this function doesn't include the "2" of the package name. Let's go ahead and try using this function on our penguins data. +::: ```{r ggplot-raw} ggplot(penguins) ``` +::: {.narration} You'll notice that the `ggplot()` function doesn't actually do much by itself. Here, we provide it with the penguins dataset, but the result looks like someone started making a plot and then stopped after the first step of making the rectangle for the plot area. This is because `ggplot2` is designed around the "grammar of graphics". Therefore, it expects you to build a sentence-like structure out of its functions. A single word (i.e., the call to `ggplot()` above) doesn't make much of a sentence, so let's start building up a real sentence. By using the `ggplot()` function, we are essentially stating that we are beginning a plotting "sentence". We then combine this with other "words" (function calls) using the `+` operator. The next component you usually want to specify in a `ggplot` "sentence" is our "aesthetic" mappings. These specify the columns of the dataset that correspond to each axis of the plot, including the x/y axes, but also the axes of color, shape, etc. We do this with the `aes()` function: +::: ```{r ggplot-aes} ggplot(penguins) + aes(x = body_mass, y = flipper_len) ``` +::: {.narration} Hey, it's starting to look like a plot now! Except there isn't any actual data being plotted. Let's fix that. We'll start off with a simple scatter plot by using the `geom_point()` function: +::: ```{r geom_point} ggplot(penguins) + @@ -62,7 +79,9 @@ ggplot(penguins) + geom_point() ``` +::: {.narration} And there we go! You'll notice that with just a few lines, we've already made a pretty nice visualization of this penguin data. `ggplot2` does most of the work for us once we specify our dataset and our `x` and `y` variables. +::: :::: {.callout-note} ## Missing data @@ -77,7 +96,9 @@ penguins <- penguins |> ::: :::: +::: {.narration} Now, let's go a step further and color the points by another variable (e.g., the island of the penguins). With `ggplot2`, all that requires is specifying another aesthetic: +::: ```{r geom_point-color} ggplot(penguins) + @@ -85,9 +106,11 @@ ggplot(penguins) + geom_point() ``` +::: {.narration} Notice that `ggplot2` comes with its own default color scheme. However, I would strongly discourage you from using the default colors, especially as the number of categories increases (with only 3 categories here it isn't too bad). Let's try out some of the more accessible color palettes that are also available in R. First, let's try one of the [`viridis`](https://cran.r-project.org/web/packages/viridis/vignettes/intro-to-viridis.html) color palettes. Since this palette is included in `ggplot2`, all we need to do is add the proper "scale" to our `ggplot()` call. Scales tell ggplot how to handle a particular aesthetic, and are usually of the form `scale_[aesthetic]_[type]()`. +::: ```{r viridis} ggplot(penguins) + @@ -96,7 +119,9 @@ ggplot(penguins) + scale_color_viridis_d(end = 0.7) # avoid yellow at the end of the palette ``` +::: {.narration} Now let's try one of the [brewer color palettes](https://r-graph-gallery.com/38-rcolorbrewers-palettes.html). +::: ```{r brewer} ggplot(penguins) + @@ -105,7 +130,9 @@ ggplot(penguins) + scale_color_brewer(palette = "Set1") ``` +::: {.narration} Outside of color, there are many other [aspects of the graph](https://ggplot2.tidyverse.org/articles/ggplot2-specs.html) that we can modify using aesthetics and "scale"s. For example, we can modify the shape of the points: +::: ```{r shape-aes} ggplot(penguins) + @@ -129,7 +156,9 @@ ggplot(penguins) + ``` ::: +::: {.narration} And the x/y axes: +::: ```{r axis-scales} ggplot(penguins) + @@ -142,9 +171,11 @@ ggplot(penguins) + scale_y_continuous(name = "Flipper Length (millimeters)") ``` -# Theming +## Theming +::: {.narration} The last basic thing you might want to do with `ggplot2` is modify the style of the visualization. This is extremely customizable, but the first place to start is with a [built-in theme](https://ggplot2.tidyverse.org/reference/ggtheme.html). I personally prefer the classic theme, which looks very similar to base R plots: +::: ```{r theme_classic} ggplot(penguins) + @@ -158,7 +189,9 @@ ggplot(penguins) + theme_classic() ``` +::: {.narration} Using this built-in theme has changed many visual aspects of the graph, including changing the plot background color, adding axis lines, and removing the internal grid lines. If you look very closely, however, the axis tick labels are still a slight grey. We can use the `theme()` function to further customize the appearance and change this. In this case, we'll make the axis text elements have a black color instead of the default gray. +::: ```{r theming} ggplot(penguins) + @@ -173,19 +206,25 @@ ggplot(penguins) + theme(axis.text = element_text(color = "black")) ``` +::: {.narration} And there we have it! With just 10 lines we've created what I would say is a publication quality graph! `ggplot` does a lot of the tedious work for us, giving us time to focus on the more important aspects, such as labeling and color. Admittedly, I've spent a LOT of time on these aspects in the past... More information about all of the hierarchical theme components that you can customize is available [here](https://ggplot2.tidyverse.org/reference/theme.html). In order to change many of these components, you need to use theme elements like we did above with `element_text()`. That and other theme elements are documented [here](https://ggplot2.tidyverse.org/reference/element.html). +::: -# More complex features +## More complex features -## Other layers +### Other layers +::: {.narration} There are many other [types of plots](https://ggplot2.tidyverse.org/reference/#layers) that we can make with `ggplot2`. +::: -### Histograms +#### Histograms +::: {.narration} We can visualize the density of values for a single variable with a histogram: +::: ```{r geom_histogram} ggplot(penguins) + @@ -207,9 +246,11 @@ ggplot(penguins) + ``` ::: -### Boxplots and Violin Plotss +#### Boxplots and Violin Plotss +::: {.narration} We can visualize the density of values for a single variable across a discrete variable with boxplots or violin plots: +::: ```{r geom_boxplot} ggplot(penguins) + @@ -232,9 +273,11 @@ ggplot(penguins) + Note that many of these "geom"s have lots of options. For example, here we've decided to `scale` all of the violin plots to the same width and to draw the quartiles on them (mimicking the boxplots above). You can see all of the options for a geom by checking out it's help page (`?geom_violin`) or on the ggplot [website](https://ggplot2.tidyverse.org/reference/geom_violin.html). ::: -### 2D Contours +#### 2D Contours +::: {.narration} We can also visualize the density of values across two continuous variables using a 2D contour: +::: ```{r geom_density_2d} ggplot(penguins) + @@ -243,7 +286,10 @@ ggplot(penguins) + theme_classic() + theme(axis.text = element_text(color = "black")) ``` + +::: {.narration} Note that sometimes you may need to expand the axes a little bit to better show the contours: +::: ```{r geom_density_2d_expand} ggplot(penguins) + @@ -254,9 +300,11 @@ ggplot(penguins) + theme(axis.text = element_text(color = "black")) ``` -### Time Series +#### Time Series +::: {.narration} Since there isn't really any time series data in the penguins dataset, we'll take a quick detour and use the built-in `economics` dataset to explore visualizing a time series. In this case, we are looking at unemployment over time: +::: ```{r geom_line} ggplot(economics, aes(x = date, y = unemploy)) + @@ -265,7 +313,9 @@ ggplot(economics, aes(x = date, y = unemploy)) + theme(axis.text = element_text(color = "black")) ``` +::: {.narration} `geom_path()` lets you explore how two variables are related over time. For example, unemployment and personal savings rate: +::: ```{r geom_path} ggplot(economics, aes(x = unemploy / pop, y = psavert)) + @@ -279,9 +329,11 @@ ggplot(economics, aes(x = unemploy / pop, y = psavert)) + Note how we've used multiple columns of the data to define the x-axis here. You can use any sort of mathematical operators to combine multiple columns into a single aesthetic, as long as you are doing row-wise math. ::: -## Combining layers +### Combining layers +::: {.narration} We can also combine multiple layers to show the same data in different ways in the same plot. For example, we could show the raw data for the above contour plot in addition to the contours: +::: ```{r combined-layers} ggplot(penguins) + @@ -301,9 +353,11 @@ ggplot(penguins) + When combining layers, the layers are added to the plot in order, so in this case the points are the bottom layer and the contour lines are the top layer. We changed the alpha of the middle layer to prevent the points from being blocked. I've also used the `coord_cartesian()` function to remove the default axis expansion. This way the background color reaches both axes and doesn't have a white gap. ::: -## Facetting +### Facetting +::: {.narration} Let's take our scatterplot example from earlier: +::: ```{r scatter-full} ggplot(penguins) + @@ -318,7 +372,9 @@ ggplot(penguins) + theme(axis.text = element_text(color = "black")) ``` +::: {.narration} Now, what if we wanted to also split the data by the species of the penguins? We're already using color and shape, so what other aesthetic could we use? We could possible use some shapes that have both a fill and outline color, but that sounds messy. Instead of using another aesthetic, we could also use a `facet`. This splits the chart into multiple panels: +::: ```{r scatter-facet} ggplot(penguins) + @@ -334,7 +390,9 @@ ggplot(penguins) + theme(axis.text = element_text(color = "black")) ``` +::: {.narration} We can get even crazier by faceting by multiple variables: +::: ```{r scatter-facet-grid} ggplot(penguins) + @@ -350,16 +408,23 @@ ggplot(penguins) + theme(axis.text = element_text(color = "black")) ``` +::: {.narration} OK, maybe we've gone a little too far here, but you get the picture! +::: -# Combining plots +## Combining plots + +::: {.narration} When publishing results, often we need to combine multiple figures into a single visualization. There are lots of packages for accomplishing this (even my own [deeptime](https://williamgearty.com/deeptime) package has some functionality for it), but today we'll check out the [`patchwork`](https://patchwork.data-imaginist.com/) package which extends the "grammar of graphics" to combining plots (you may need to install it if you haven't done so already). +::: ```{r patchwork} library(patchwork) ``` +::: {.narration} First, let's go ahead and make some plots. Instead of plotting them, though, we'll save them as objects in our environment. Note that when you save a plot to an object it isn't displayed in the "Plots" tab. +::: ```{r plot-objects} g1 <- ggplot(penguins) + @@ -380,13 +445,17 @@ g2 <- ggplot(penguins) + theme(axis.text = element_text(color = "black")) ``` +::: {.narration} Now, in order to combine these, all we need to do is combine them using the `+` operator, like we did with the individual elements of the plots. +::: ```{r add-plots} g1 + g2 ``` +::: {.narration} You can see that `patchwork` does all of the work for us, lining up the different components of the plots. If we have more plots to combine, we can then use the `|` (side-by-side) and `/` (above-and-below) operators to make more complex arrangements of plots. +::: ```{r patchwork-complex} g3 <- ggplot(penguins) + @@ -398,15 +467,18 @@ g3 <- ggplot(penguins) + (g1 | g2) / g3 ``` +::: {.narration} There's a lot more you can do with `patchwork`, including adjusting the widths and heights, but we'll stop here for now. +::: + +## Saving plots -# Saving plots +::: {.narration} The last big thing you'll need to know about plotting is how to save your plots. `ggplot2` comes with a nifty `ggsave()` function which you can use to save your plots in a number of different formats. Here we'll save our most recent combined plot as both a PDF and a JPEG. The former is a vector format, meaning all of the elements of the figure as geometric shapes, and, because of this, none of the data is lost (aka "lossless"). The latter is a raster format, meaning the figure is converted to a 2-D array of colored pixels of a desired size, and because of this, some of the data is lost in the process (aka "lossy"). This results in the pixellation that you see when you zoom in on a JPEG. `ggsave()` detects what format you want based on the file extension, so we just need to specify the file location and the object to be saved (and optionally the dimensions of the output file). It's usually a good idea to keep all figures in their own folder (which was already created for you). +::: ```{r ggsave, eval=FALSE} gg <- (g1 | g2) / g3 ggsave("figures/penguins_1.pdf", gg, height = 10, width = 10) ggsave("figures/penguins_1.jpg", gg, height = 10, width = 10) ``` - -Now we'll once again use GitHub Desktop to commit these files and push them to our GitHub repository. You should see all of the files that have been modified since we last committed. In this case, you should see your code file (modified from our last session) and your two new figures. Make sure the checkboxes are checked for these three files, write a succinct commit summary, then click the Commit button, then the Push button. And that's that! diff --git a/ggplot/index.qmd.orig b/ggplot/index.qmd.orig new file mode 100644 index 0000000..69ae6b3 --- /dev/null +++ b/ggplot/index.qmd.orig @@ -0,0 +1,410 @@ +--- +title: "Plotting in R with ggplot2" +description: "Visualizing your data with the grammar of graphics" +author: "Will Gearty" +date: "2026-06-12" +categories: [r, tidyverse, dataviz] +difficulty: Beginner +image: images/2d_density.png +format: + html: default + revealjs: + output-file: index-slides.html +execute: + warning: false + message: false + freeze: auto +editor: + markdown: + wrap: 72 +--- + +## Introduction + +While you can make plots with just the packages that come bundled with base R, many R users make their visualizations entirely using the [`ggplot2`](https://ggplot2.tidyverse.org/index.html) package and an [ecosystem of packages](https://exts.ggplot2.tidyverse.org/gallery/) designed around it. + +```{r ggplot2} +# load the ggplot2 package +library(ggplot2) +``` + +As with the previous sessions, we'll be using the Palmer penguins dataset. While we built our own combined dataset in the introduction session, now we're going to use the built-in cleaned dataset. First, let's load the dataset. Then let's inspect the data using the `glimpse()` function. + +```{r palmerpenguins} +data(penguins) +dplyr::glimpse(penguins) +``` + +As we discovered before, this dataset includes many different measurements for individual penguins from three different studies. The studies cover both sexes of three different species of penguins from three different islands in the Palmer Archipelago. + +## The ggplot2 basics + +The most important function in the `ggplot2` package is `ggplot()`. Note that this function doesn't include the "2" of the package name. Let's go ahead and try using this function on our penguins data. + +```{r ggplot-raw} +ggplot(penguins) +``` + +You'll notice that the `ggplot()` function doesn't actually do much by itself. Here, we provide it with the penguins dataset, but the result looks like someone started making a plot and then stopped after the first step of making the rectangle for the plot area. This is because `ggplot2` is designed around the "grammar of graphics". Therefore, it expects you to build a sentence-like structure out of its functions. A single word (i.e., the call to `ggplot()` above) doesn't make much of a sentence, so let's start building up a real sentence. + +By using the `ggplot()` function, we are essentially stating that we are beginning a plotting "sentence". We then combine this with other "words" (function calls) using the `+` operator. The next component you usually want to specify in a `ggplot` "sentence" is our "aesthetic" mappings. These specify the columns of the dataset that correspond to each axis of the plot, including the x/y axes, but also the axes of color, shape, etc. We do this with the `aes()` function: + +```{r ggplot-aes} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len) +``` + +Hey, it's starting to look like a plot now! Except there isn't any actual data being plotted. Let's fix that. We'll start off with a simple scatter plot by using the `geom_point()` function: + +```{r geom_point} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len) + + geom_point() +``` + +And there we go! You'll notice that with just a few lines, we've already made a pretty nice visualization of this penguin data. `ggplot2` does most of the work for us once we specify our dataset and our `x` and `y` variables. + +:::: {.callout-note} +## Missing data +You may have noticed a warning that some rows of the dataset contain missing values. There are two penguins without any measurements in the dataset. How might we remove them using `dplyr` so we don't get this warning over and over? + +::: {.callout-caution collapse="true" appearance="simple" icon="false"} +##### Solution +```{r} +penguins <- penguins |> + dplyr::filter(!is.na(body_mass)) +``` +::: +:::: + +Now, let's go a step further and color the points by another variable (e.g., the island of the penguins). With `ggplot2`, all that requires is specifying another aesthetic: + +```{r geom_point-color} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, color = island) + + geom_point() +``` + +Notice that `ggplot2` comes with its own default color scheme. However, I would strongly discourage you from using the default colors, especially as the number of categories increases (with only 3 categories here it isn't too bad). Let's try out some of the more accessible color palettes that are also available in R. + +First, let's try one of the [`viridis`](https://cran.r-project.org/web/packages/viridis/vignettes/intro-to-viridis.html) color palettes. Since this palette is included in `ggplot2`, all we need to do is add the proper "scale" to our `ggplot()` call. Scales tell ggplot how to handle a particular aesthetic, and are usually of the form `scale_[aesthetic]_[type]()`. + +```{r viridis} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, color = island) + + geom_point() + + scale_color_viridis_d(end = 0.7) # avoid yellow at the end of the palette +``` + +Now let's try one of the [brewer color palettes](https://r-graph-gallery.com/38-rcolorbrewers-palettes.html). + +```{r brewer} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, color = island) + + geom_point() + + scale_color_brewer(palette = "Set1") +``` + +Outside of color, there are many other [aspects of the graph](https://ggplot2.tidyverse.org/articles/ggplot2-specs.html) that we can modify using aesthetics and "scale"s. For example, we can modify the shape of the points: + +```{r shape-aes} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, + color = island, shape = sex) + + geom_point() + + scale_color_brewer(palette = "Set1") +``` + +::: {.callout-note} +### More missing data +Urgh, more missing data! It appears that nine of the penguins are missing sex identification data. We can ignore the warning message, but that extra "NA" category in the legend is quite annoying. We can remove this category from the legend using `na.translate = FALSE` within `scale_shape_discrete()` (you could also use `scale_shape_manual()` if you wanted to supply your own shapes): + +```{r na.translate} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, + color = island, shape = sex) + + geom_point() + + scale_color_brewer(palette = "Set1") + + scale_shape_discrete(na.translate = FALSE) +``` +::: + +And the x/y axes: + +```{r axis-scales} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, + color = island, shape = sex) + + geom_point() + + scale_color_brewer(palette = "Set1") + + scale_shape_discrete(na.translate = FALSE) + + scale_x_continuous(name = "Body Mass (grams)") + + scale_y_continuous(name = "Flipper Length (millimeters)") +``` + +## Theming + +The last basic thing you might want to do with `ggplot2` is modify the style of the visualization. This is extremely customizable, but the first place to start is with a [built-in theme](https://ggplot2.tidyverse.org/reference/ggtheme.html). I personally prefer the classic theme, which looks very similar to base R plots: + +```{r theme_classic} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, + color = island, shape = sex) + + geom_point() + + scale_color_brewer(palette = "Set1") + + scale_shape_discrete(na.translate = FALSE) + + scale_x_continuous(name = "Body Mass (grams)") + + scale_y_continuous(name = "Flipper Length (millimeters)") + + theme_classic() +``` + +Using this built-in theme has changed many visual aspects of the graph, including changing the plot background color, adding axis lines, and removing the internal grid lines. If you look very closely, however, the axis tick labels are still a slight grey. We can use the `theme()` function to further customize the appearance and change this. In this case, we'll make the axis text elements have a black color instead of the default gray. + +```{r theming} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, + color = island, shape = sex) + + geom_point() + + scale_color_brewer(palette = "Set1") + + scale_shape_discrete(na.translate = FALSE) + + scale_x_continuous(name = "Body Mass (grams)") + + scale_y_continuous(name = "Flipper Length (millimeters)") + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +And there we have it! With just 10 lines we've created what I would say is a publication quality graph! `ggplot` does a lot of the tedious work for us, giving us time to focus on the more important aspects, such as labeling and color. Admittedly, I've spent a LOT of time on these aspects in the past... + +More information about all of the hierarchical theme components that you can customize is available [here](https://ggplot2.tidyverse.org/reference/theme.html). In order to change many of these components, you need to use theme elements like we did above with `element_text()`. That and other theme elements are documented [here](https://ggplot2.tidyverse.org/reference/element.html). + +## More complex features + +### Other layers + +There are many other [types of plots](https://ggplot2.tidyverse.org/reference/#layers) that we can make with `ggplot2`. + +#### Histograms + +We can visualize the density of values for a single variable with a histogram: + +```{r geom_histogram} +ggplot(penguins) + + aes(x = body_mass, fill = species) + + geom_histogram() + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +::: {.callout-note} +Histograms don't require a y-axis aesthetic by default. The counts are tabulated for you. If you specify a "fill" aesthetic, the default is to stack the bars which can sometimes be a bit misleading. You can also dodge them to fix this: + +```{r hist-dodge} +ggplot(penguins) + + aes(x = body_mass, fill = species) + + geom_histogram(position = "dodge") + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` +::: + +#### Boxplots and Violin Plotss + +We can visualize the density of values for a single variable across a discrete variable with boxplots or violin plots: + +```{r geom_boxplot} +ggplot(penguins) + + aes(x = island, y = bill_len) + + geom_boxplot() + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +```{r geom_violin} +ggplot(penguins) + + aes(x = island, y = bill_len) + + geom_violin(scale = "width", draw_quantiles = c(0.25, 0.5, 0.75)) + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +::: {.callout-note} +#### Geom options +Note that many of these "geom"s have lots of options. For example, here we've decided to `scale` all of the violin plots to the same width and to draw the quartiles on them (mimicking the boxplots above). You can see all of the options for a geom by checking out it's help page (`?geom_violin`) or on the ggplot [website](https://ggplot2.tidyverse.org/reference/geom_violin.html). +::: + +#### 2D Contours + +We can also visualize the density of values across two continuous variables using a 2D contour: + +```{r geom_density_2d} +ggplot(penguins) + + aes(x = bill_len, y = bill_dep) + + geom_density_2d(linewidth = 0.25, colour = "black") + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` +Note that sometimes you may need to expand the axes a little bit to better show the contours: + +```{r geom_density_2d_expand} +ggplot(penguins) + + aes(x = bill_len, y = bill_dep) + + geom_density_2d(linewidth = 0.25, colour = "black") + + scale_y_continuous(limits = c(12.5, NA)) + # NA means "don't change" + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +#### Time Series + +Since there isn't really any time series data in the penguins dataset, we'll take a quick detour and use the built-in `economics` dataset to explore visualizing a time series. In this case, we are looking at unemployment over time: + +```{r geom_line} +ggplot(economics, aes(x = date, y = unemploy)) + + geom_line() + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +`geom_path()` lets you explore how two variables are related over time. For example, unemployment and personal savings rate: + +```{r geom_path} +ggplot(economics, aes(x = unemploy / pop, y = psavert)) + + geom_path(aes(colour = as.numeric(date))) + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +::: {.callout-note} +#### Multiple columns for individual aesthetics +Note how we've used multiple columns of the data to define the x-axis here. You can use any sort of mathematical operators to combine multiple columns into a single aesthetic, as long as you are doing row-wise math. +::: + +### Combining layers + +We can also combine multiple layers to show the same data in different ways in the same plot. For example, we could show the raw data for the above contour plot in addition to the contours: + +```{r combined-layers} +ggplot(penguins) + + aes(x = bill_len, y = bill_dep) + + geom_point() + + geom_density_2d_filled(alpha = 0.5) + + geom_density_2d(linewidth = 0.25, colour = "black") + + scale_x_continuous(limits = c(30, 60)) + + scale_y_continuous(limits = c(12, 23)) + + coord_cartesian(expand = FALSE) + # don't expand the axes so the background is all filled + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +::: {.callout-note} +### Layer order +When combining layers, the layers are added to the plot in order, so in this case the points are the bottom layer and the contour lines are the top layer. We changed the alpha of the middle layer to prevent the points from being blocked. I've also used the `coord_cartesian()` function to remove the default axis expansion. This way the background color reaches both axes and doesn't have a white gap. +::: + +### Facetting + +Let's take our scatterplot example from earlier: + +```{r scatter-full} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, + color = island, shape = sex) + + geom_point() + + scale_color_brewer(palette = "Set1") + + scale_shape_discrete(na.translate = FALSE) + + scale_x_continuous(name = "Body Mass (grams)") + + scale_y_continuous(name = "Flipper Length (millimeters)") + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +Now, what if we wanted to also split the data by the species of the penguins? We're already using color and shape, so what other aesthetic could we use? We could possible use some shapes that have both a fill and outline color, but that sounds messy. Instead of using another aesthetic, we could also use a `facet`. This splits the chart into multiple panels: + +```{r scatter-facet} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, + color = island, shape = sex) + + geom_point() + + scale_color_brewer(palette = "Set1") + + scale_shape_discrete(na.translate = FALSE) + + scale_x_continuous(name = "Body Mass (grams)") + + scale_y_continuous(name = "Flipper Length (millimeters)") + + facet_wrap(vars(species), ncol = 1) + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +We can get even crazier by faceting by multiple variables: + +```{r scatter-facet-grid} +ggplot(penguins) + + aes(x = body_mass, y = flipper_len, + color = island, shape = sex) + + geom_point() + + scale_color_brewer(palette = "Set1") + + scale_shape_discrete(na.translate = FALSE) + + scale_x_continuous(name = "Body Mass (grams)") + + scale_y_continuous(name = "Flipper Length (millimeters)") + + facet_grid(rows = vars(species), cols = vars(year)) + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +OK, maybe we've gone a little too far here, but you get the picture! + +## Combining plots +When publishing results, often we need to combine multiple figures into a single visualization. There are lots of packages for accomplishing this (even my own [deeptime](https://williamgearty.com/deeptime) package has some functionality for it), but today we'll check out the [`patchwork`](https://patchwork.data-imaginist.com/) package which extends the "grammar of graphics" to combining plots (you may need to install it if you haven't done so already). + +```{r patchwork} +library(patchwork) +``` + +First, let's go ahead and make some plots. Instead of plotting them, though, we'll save them as objects in our environment. Note that when you save a plot to an object it isn't displayed in the "Plots" tab. + +```{r plot-objects} +g1 <- ggplot(penguins) + + aes(x = body_mass, y = flipper_len) + + geom_point() + + theme_classic() + + theme(axis.text = element_text(color = "black")) + +g2 <- ggplot(penguins) + + aes(x = bill_len, y = bill_dep) + + geom_point() + + geom_density_2d_filled(alpha = 0.5) + + geom_density_2d(linewidth = 0.25, colour = "black") + + scale_x_continuous(limits = c(30, 60)) + + scale_y_continuous(limits = c(12, 23)) + + coord_cartesian(expand = FALSE) + + theme_classic() + + theme(axis.text = element_text(color = "black")) +``` + +Now, in order to combine these, all we need to do is combine them using the `+` operator, like we did with the individual elements of the plots. + +```{r add-plots} +g1 + g2 +``` + +You can see that `patchwork` does all of the work for us, lining up the different components of the plots. If we have more plots to combine, we can then use the `|` (side-by-side) and `/` (above-and-below) operators to make more complex arrangements of plots. + +```{r patchwork-complex} +g3 <- ggplot(penguins) + + aes(x = island, y = bill_len) + + geom_boxplot() + + theme_classic() + + theme(axis.text = element_text(color = "black")) + +(g1 | g2) / g3 +``` + +There's a lot more you can do with `patchwork`, including adjusting the widths and heights, but we'll stop here for now. + +## Saving plots +The last big thing you'll need to know about plotting is how to save your plots. `ggplot2` comes with a nifty `ggsave()` function which you can use to save your plots in a number of different formats. Here we'll save our most recent combined plot as both a PDF and a JPEG. The former is a vector format, meaning all of the elements of the figure as geometric shapes, and, because of this, none of the data is lost (aka "lossless"). The latter is a raster format, meaning the figure is converted to a 2-D array of colored pixels of a desired size, and because of this, some of the data is lost in the process (aka "lossy"). This results in the pixellation that you see when you zoom in on a JPEG. `ggsave()` detects what format you want based on the file extension, so we just need to specify the file location and the object to be saved (and optionally the dimensions of the output file). It's usually a good idea to keep all figures in their own folder (which was already created for you). + +```{r ggsave, eval=FALSE} +gg <- (g1 | g2) / g3 +ggsave("figures/penguins_1.pdf", gg, height = 10, width = 10) +ggsave("figures/penguins_1.jpg", gg, height = 10, width = 10) +``` diff --git a/sync_filter.r b/sync_filter.r new file mode 100644 index 0000000..bb8f72a --- /dev/null +++ b/sync_filter.r @@ -0,0 +1,36 @@ +#!/usr/bin/env Rscript + +# Copy the Lua filter (web_and_slides.lua) into every module as +# web_and_slides_autogenerated.lua. +# +# Register this as `pre-render` in _quarto.yml to run it before every local render. +# +# Usage: Rscript sync_filter.r (run from the repo root) + +source_lua <- "web_and_slides.lua" +copy_name <- "web_and_slides_autogenerated.lua" # also referenced by web_and_slides.r + +if (!file.exists(source_lua)) { + stop("Cannot find ", source_lua, " in ", getwd(), + " -- run this from the repo root") +} + +banner <- c( + paste0("-- ", copy_name, " -- AUTOGENERATED, DO NOT EDIT."), + paste0("-- Copied from the repo root's ", source_lua, " by sync_filter.r (see _quarto.yml)."), + "-- Edit that file instead; this copy exists so the module is self-contained.", + "--" +) +content <- c(banner, readLines(source_lua, warn = FALSE)) + +# modules are the top-level directories holding an index.qmd +modules <- dirname(list.files(pattern = "^index\\.qmd$", recursive = TRUE)) +modules <- setdiff(modules, ".") +if (length(modules) == 0) message("sync_filter: no modules found") + +for (m in modules) { + dest <- file.path(m, copy_name) + if (file.exists(dest) && identical(readLines(dest, warn = FALSE), content)) next + writeLines(content, dest) + message("sync_filter: wrote ", dest) +} diff --git a/web_and_slides.lua b/web_and_slides.lua new file mode 100644 index 0000000..7a78c9b --- /dev/null +++ b/web_and_slides.lua @@ -0,0 +1,223 @@ +-- web_and_slides.lua +-- Combined filter for dual html + revealjs Quarto modules: +-- * Div : `::: {.narration}` -> speaker notes on revealjs, plain prose elsewhere. +-- `::: {.slides-only}` -> revealjs slides only: dropped elsewhere +-- (equivalent to `::: {.content-visible when-format="revealjs"}`) +-- `::: {.html-only}` -> html only: dropped on revealjs slides +-- (equivalent to `::: {.content-visible when-format="html"}`) +-- * Pandoc : on revealjs, headings below the slide level are promoted so each +-- becomes its own slide, the slide closes after each plot (so no two +-- plots share a slide and a plot's following prose stays with the +-- next plot -- unless only prose remains before the next heading, in +-- which case it stays put rather than making a blank slide), and each +-- callout is unwrapped onto its own slide titled by its own heading +-- (no box, so any figure inside hoists and stretches; an untitled +-- callout keeps the section title). Split slides repeat the current +-- heading so they keep a title. Finally, any slide with 2+ cells is +-- expanded into an auto-animate build-up (one step per cell, cells +-- accumulating, notes advancing) so each step is a real slide whose +-- figure still hoists and auto-stretches. Other formats untouched. +-- Register at the `pre-ast` stage so callouts are still plain divs here (Quarto +-- normalizes them into custom AST nodes after that point). + +function Div(el) + if el.classes:includes("slides-only") then + if not quarto.doc.is_format("revealjs") then + return {} -- slides only: nothing on the website + end + return el.content + end + if el.classes:includes("html-only") then + if not quarto.doc.is_format("html") then + return {} -- html only: nothing on the slides + end + return el.content + end + if not el.classes:includes("narration") then + return nil -- leave every other div alone + end + if quarto.doc.is_format("revealjs") then + el.classes = el.classes:filter(function(c) return c ~= "narration" end) + el.classes:insert("notes") -- Pandoc emits