Visualization of highly-multiplexed imaging data with cytomapper

class: center, middle, inverse, title-slide

# Visualization of highly-multiplexed imaging data with <code>cytomapper</code>
### Nils Eling
### Department of Quantitative Biomedicine, University of Zurich
### Institute for Molecular Health Sciences, ETH Zurich
### 09/07/2021

---

background-image: url(imgs/cytomapper_sticker.png)
background-size: 150px
background-position: 95% 10%

## The `cytomapper` R/Bioconductor package

Contains three broad functionalities:

* Visualization of pixel-intensities as composites of up to six channels (`plotPixels`)

* Visualization of cell-specific features on segmentation masks (`plotCells`)

* Interactive gating of cells and visualization of gated cells on images (`cytomapperShiny`)

### Installation

Via Bioconductor (release version):

```r
BiocManager::install("cytomapper")
```

or from Github (development version):

```r
remotes::install_github("BodenmillerGroup/cytomapper", build_vignettes = TRUE)
```

---

## Data structure

.pull-left[

* `cytomapper` uses a `SingleCellExperiment` data container to store cell-specific metadata and intensity features

* the new class `cytomapper::CytoImageList` stores individual `EBImage::Image` objects in form of a `S4Vectors::SimpleList` container

* visualization of cell-specific metadata is possible by linking a `CytoImageList` and `SingleCellExperiment` object via the function parameters `img_id` and `cell_id`

]

.pull-right[

![cytomapper overview figure. A) The plotCells function combines a SingleCellExperiment and CytoImageList object to visualize marker expression or cell-specific metadata on segmentation masks. B) The plotPixels function requires a CytoImageList object to visualize the combined expression of up to six markers as composite images](imgs/Overview.png)

]

<div class="my-footer" style="height=100px;"><a href="https://bioconductor.org/packages/release/bioc/html/SingleCellExperiment.html">https://bioconductor.org/packages/release/bioc/html/SingleCellExperiment.html</a><br>
<a href="https://bioconductor.org/packages/release/bioc/html/EBImage.html">https://bioconductor.org/packages/release/bioc/html/EBImage.html</a><br>
<a href="https://bioconductor.org/packages/release/bioc/html/S4Vectors.html">https://bioconductor.org/packages/release/bioc/html/S4Vectors.html</a><br>
<a href="https://github.com/BodenmillerGroup/ImcSegmentationPipeline">https://github.com/BodenmillerGroup/ImcSegmentationPipeline</a></div>

???

In the displayed framework, cells are defined as pixel sets with the same id on the segmentation mask. These masks and their quantification are done in pre-processing steps that are not mentioned in detail here put a link to a segmentation pipeline can be found in the footer.

---

## Reading in the images

The `cytomapper::loadImages` function reads in multi-channel images and segmentation masks into `CytoImageList` objects.

```r
library(cytomapper)

# Read in 32-bit multi-channel images
(images <- loadImages("../data/images/", pattern = ".tiff"))
```

```
## CytoImageList containing 3 image(s)
## names(3): E30_a0_full_clean G23_a0_full_clean J01_a0_full_clean 
## Each image contains 38 channel(s)
```

```r
# Read in 16-bit unsigned integer segmentation masks
(masks <- loadImages("../data/masks/", pattern = ".tiff", as.is = TRUE))
```

```
## CytoImageList containing 3 image(s)
## names(3): E30_a0_full_mask G23_a0_full_mask J01_a0_full_mask 
## Each image contains 1 channel
```

<div class="my-footer" style="height=50px;"><a href="https://github.com/BodenmillerGroup/cytomapper_demos">https://github.com/BodenmillerGroup/cytomapper_demos</a></div>

???

The loadImages function reads in .tiff, .png or .jpg images and formats them into a CytoImageList container. The as.is parameter needs to be set to TRUE to correctly read in 16-bit unsigned integer images. The multi-channel images and segmentation masks are provided in the BodenmillerGroup/cytomapper_demos Github repository.

---

## Reading in the single-cell data

Cell-specific metadata and summarized intensity features are stored in form of a `SingleCellExperiment` object.

```r
# Read in single-cell data
(sce <- readRDS("../data/sce.rds"))
```

```
## class: SingleCellExperiment 
## dim: 38 5819 
## metadata(0):
## assays(2): counts exprs
## rownames(38): H3 SMA ... Ir191 Ir193
## rowData names(15): TubeNb MetalTag ... miCAT2 miCAT
## colnames(5819): E30_1 E30_2 ... J01_2173 J01_2174
## colData names(26): slide id ... Ethnicity BMI
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
```

<div class="my-footer" style="height=50px;"><a href="https://github.com/BodenmillerGroup/cytomapper_demos">https://github.com/BodenmillerGroup/cytomapper_demos</a></div>

???

In the BodenmillerGroup/cytomapper_demos Github repository, you can also find an .rds file containing a SingleCellExperiment object that holds the mean channel intensities per cell and all associated metadata (e.g. cell-type label). Cell-specific intensities are stored in the `counts(sce)` slot; cell-specific metadata are stored in the `colData(sce)` slot and marker-specific metadata are stored in `rowData(sce)`.

---

## Formatting the data

Prior to visualization, the channel names and image-specific metadata should be set.

```r
# Add channel names
channelNames(images) <- rownames(sce)

# Add image name to metadata
(mcols(images) <- mcols(masks) <- DataFrame(ImageName = c("E30", "G23", "J01")))
```

```
## DataFrame with 3 rows and 1 column
##     ImageName
##   <character>
## 1         E30
## 2         G23
## 3         J01
```

???

We will need to set the channel names of the images via the `channelNames` getter/setter function. The channel order here is the same as the row order of the `SingleCellExperiment` object. We will also need to synchronise the image IDs across the multi-channel images and segmentation masks by storing a DataFrame in the elementMetadata slot of the CytoImageList object.

---

## Measuring object features

The `cytomapper::measureObjects` function computes morphological (e.g. cell shape, size and location) and intensity features (default: mean intensity per channel and object/cell).

```r
sce_2 <- measureObjects(mask = masks, image = images, img_id = "ImageName")
sce_2
```

```
## class: SingleCellExperiment 
## dim: 38 5819 
## metadata(0):
## assays(1): counts
## rownames(38): H3 SMA ... Ir191 Ir193
## rowData names(0):
## colnames: NULL
## colData names(8): ImageName object_id ... m.majoraxis m.eccentricity
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
```

???

The pixel intensities per cell can be summarized in different ways (e.g. as quantiles). Furthermore, parallelization is possible by setting `BPPARAM = bpparam()`. Cell-specific morphological features are stored in `colData(sce)`.

---

## Visualize multi-channel images

.pull-left[

```r
plotPixels(
    image = images,
    colour_by = c("PIN", "CD4", "CD8a"), 
    colour = list(PIN = c("black", "yellow"),
                  CD4 = c("black", "blue"),
                  CD8a = c("black", "red")),
    bcg = list(PIN = c(0, 10, 1),
               CD4 = c(0, 8, 1),
               CD8a = c(0, 10, 1)),
    image_title = list(
        text = c("Non-diabetic",
                 "Recent onset T1D",
                 "Long duration T1D")
        ),
    scale_bar = list(
        length = 100,
        label = expression("100 " ~ mu * "m")
        ))
```

]

.pull-right[

]

???

The `plotPixels` function visualizes pixel intensities as composites of up to six markers. It requires at least the `image` parameters that accepts a `CytoImageList` object containing the individual multi-channel images. The `colour_by` parameter defines the channel names by which to colour the composite. Per channel, a colour scale is generated by setting `colour`. The bcg parameter defines the background (b), contrast (c), gamma (g) adjustment of each channel in terms of addition (b), multiplication (c) and exponentiation (g). The attributes of the image titles can be set via the parameter `image_title` and attributes of the scale bar are set via the parameter `scale_bar`.

---

## Visualize segmented cells

.pull-left[

```r
cur_sce <- sce[,sce$CellType %in% 
    c("beta", "alpha", "delta", "Tc", "Th")]
plotCells(
    mask = masks,
    object = cur_sce,
    cell_id = "CellNumber",
    img_id = "ImageName",
    colour_by = "CellType",
    image_title = list(
        text = c("Non-diabetic",
                 "Recent onset T1D",
                 "Long duration T1D"),
        colour = "black"),
    scale_bar = list(
        length = 100,
        label = expression("100 " ~ mu * "m"),
        colour = "black"),
    missing_colour = "white",
    background_colour = "gray")
```

]

.pull-right[

]

To highlight the functionality of the `plotCells` function, the `SingleCellExperiment` object is first subsetted to contain only objects of a certain type. Here, these are pancreatic islet cells and T cells. The `plotCells` function requires the `mask` parameter, which takes a `CytoImageList` object storing the segmentation masks. Via the `object` parameter, the user can supply a `SingleCellExperiment` object containing expression or metadata. Via the `cell_id`, cells contained in the `SingleCellExperiment` object are linked to their ID on the segmentation mask. Setting `img_id` will link cells in the `SingleCellExperiment` object to their corresponding images. The `missing_colour` parameter defines the colour of cells, which are not contained in the SingleCellExperiment object. Finally, the `background_colour` parameter sets the colour of the segmentation mask background.

---

## Outline cells on images

.pull-left[

```r
plotPixels(
    image = images,
    object = cur_sce,
    mask = masks,
    cell_id = "CellNumber",
    img_id = "ImageName",
    colour_by = c("PIN", "CD4", "CD8a"), 
    outline_by = "CellType",
    colour = list(PIN = c("black", "yellow"),
                  CD4 = c("black", "blue"),
                  CD8a = c("black", "red")),
    bcg = list(PIN = c(0, 10, 1),
               CD4 = c(0, 8, 1),
               CD8a = c(0, 10, 1)),
    image_title = list(text = c("Non-diabetic",
                                "Recent onset T1D",
                                "Long duration T1D")),
    scale_bar = list(length = 100,
                     label = expression("100 " ~ mu * "m")),
    thick = TRUE)
```

]

.pull-right[

]

???

Finally, `cytomapper` can be used to outline cells on composite images while colouring the outlines based on specified metadata. For this, the `plotPixels` function combines the multi-channel images, the SingleCellExperiment object and the segmentation masks. AS before, cells and images are matched via the `cell_id` and `img_id` parameters. Here, the outline colour is determined by the cells' phenotype and outlines are expaned by setting `thick = TRUE`.

---

## Acknowledgements and useful links

**Co-authors:**

Nicolas Damond, Tobias Hoch, Bernd Bodenmiller

**Publication:**

<a href="https://academic.oup.com/bioinformatics/article/36/24/5706/6050702">Eling _et al._, cytomapper: an R/Bioconductor package for visualization of highly multiplexed imaging data, Bioinformatics, 2020</a>

<a href="https://github.com/BodenmillerGroup/cytomapper_publication">https://github.com/BodenmillerGroup/cytomapper_publication</a>

**Vignettes:**

<a href="https://www.bioconductor.org/packages/release/bioc/vignettes/cytomapper/inst/doc/cytomapper.html">Visualization of imaging cytometry data in R</a>

<a style="color=blue;" href="https://www.bioconductor.org/packages/release/bioc/vignettes/cytomapper/inst/doc/cytomapper_ondisk.html">On disk storage of images</a>

**Related packages:**

<a href="https://bioconductor.org/packages/release/data/experiment/html/imcdatasets.html">imcdatasets</a>

<a href="https://github.com/BodenmillerGroup/imcRtools">imcRtools</a>

???

With this, I want to thank my colleagues who helped me with the work: Nicolas Damond, Tobias Hoch and Bernd Bodenmiller and my funders EMBO and Marie Curie. If you want to check out cytomapper have a look at the publication and the package vignettes. Specifically how to read in and store images on disk. We have also been working on extending the multiplexed imaging capabilities of Bioconductor by setting up the imcdatasets package which stores publically available imaging mass cytometry datasets and imcRtools which provides functions for multiplexed imaging data analysis - but which is still under development.

---

## Session Info

```r
sessioninfo::package_info() %>% filter(source == "Bioconductor")
```

```
##  package              * version date       lib source      
##  Biobase              * 2.52.0  2021-05-19 [1] Bioconductor
##  BiocGenerics         * 0.38.0  2021-05-19 [1] Bioconductor
##  BiocParallel           1.26.0  2021-05-19 [1] Bioconductor
##  cytomapper           * 1.4.1   2021-05-21 [1] Bioconductor
##  DelayedArray           0.18.0  2021-05-19 [1] Bioconductor
##  EBImage              * 4.34.0  2021-05-19 [1] Bioconductor
##  GenomeInfoDb         * 1.28.0  2021-05-19 [1] Bioconductor
##  GenomeInfoDbData       1.2.6   2021-05-29 [1] Bioconductor
##  GenomicRanges        * 1.44.0  2021-05-19 [1] Bioconductor
##  HDF5Array              1.20.0  2021-05-19 [1] Bioconductor
##  IRanges              * 2.26.0  2021-05-19 [1] Bioconductor
##  MatrixGenerics       * 1.4.0   2021-05-19 [1] Bioconductor
##  rhdf5                  2.36.0  2021-05-19 [1] Bioconductor
##  rhdf5filters           1.4.0   2021-05-19 [1] Bioconductor
##  Rhdf5lib               1.14.1  2021-06-08 [1] Bioconductor
##  S4Vectors            * 0.30.0  2021-05-19 [1] Bioconductor
##  SingleCellExperiment * 1.14.1  2021-05-21 [1] Bioconductor
##  SummarizedExperiment * 1.22.0  2021-05-19 [1] Bioconductor
##  XVector                0.32.0  2021-05-19 [1] Bioconductor
##  zlibbioc               1.38.0  2021-05-19 [1] Bioconductor
## 
## [1] /Users/nils/Library/R/x86_64/4.1/library
## [2] /Library/Frameworks/R.framework/Versions/4.1/Resources/library
```

???

Finally, here are the main Bioconductor packages used for the displayed analysis and I'm looking forward to your questions.