Exploring dimension-reduced embeddings with Sleepwalk

The Sleepwalk app

Here, we present Sleepwalk, an interactive tool that provides an intuitive solution to the task just outlined.

It works as follows: The user provides an embedding, that is, the two-dimensional coordinates output by a dimension-reducing method, as well as information on the distances between cells in feature space in some suitable metric or their coordinates in an appropriately transformed feature space. Whenever the user moves the mouse cursor over a cell, all cells are colored according to their distance to this cell in feature space, thus indicating the cell's closest neighbors with the strongest color (Fig. 2A–C). By moving the mouse over all the cells in the plot, the user can hence quickly obtain an intuitive overview of how neighborhood may have been rendered differently in different regions of the plot. Buttons are provided to adjust the color scale so that the user can choose which feature space distance should be considered as “close neighborhood” and hence given the strong (dark, green) colors.

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 2.

The Sleepwalk app, being used to explore the t-SNE rendition of the cord-blood data set from Figure 1. The plots here are snapshots of a running Sleepwalk app; a live version can be found at https://anders-biostat.github.io/sleepwalk/supplementary/. The red arrow shows the current mouse position. (A) By moving the mouse cursor through the embedding, we find, for example, that the CD4⁺ T cell cluster is very tight and homogeneous, as can be seen from the fact that all cells show a color indicating that they are all close to each other. (B) The monocyte cluster, in contrast, shows much more heterogeneity, when comparing the coloring at the same color scale: Now only a few monocytes are colored green and are hence as similar to the cell under the mouse cursor as most of the T cells were in A. (C) Placing the mouse on this small tip of the monocyte cluster reveals that the cells there are more similar to the T cells than to the other monocytes, indicating that the cluster boundary might be inaccurate in both the t-SNE rendition and the SNN clustering (Waltman and van Eck 2013) on which the Seurat workflow's cell-type assignment is based. (D) With the color scale set to a wider distance range, we can assess similarities between clusters: As expected, B cells are somewhat similar to T cells, less so to NK cells and monocytes, and distant to erythrocytes and the spiked-in mouse cells.

We invite the reader to pause here and try Sleepwalk himself or herself. At https://anders-biostat.github.io/sleepwalk/supplementary/ (and also in this paper's Supplemental HTML), live interactive Sleepwalk renditions of Figure 2 (and also of all the subsequent figures discussed below) are given. The Sleepwalk app runs in any Javascript-enabled web browser; that is, it suffices to open the page or file in a browser, without the need to install anything.

Exploring an embedding

Sleepwalk makes aspects visible that are not apparent from a dimension reduction alone. For example, the two large clusters under the cursor in Figure 2, A and B, have quite different characteristics. In the T cell cluster (Fig. 2A), most of the cells are very close to each other: The cluster shows up as a large green cloud no matter where one points the mouse. The monocyte cluster (Fig. 2B), however, spreads over larger distances: Only a part shows up in green, which “follows” the mouse. In a static t-SNE plot (such as Fig. 1), this cannot be seen.

We can also check the cluster borders and discover, for instance (Fig. 2C), that some cells in the monocyte cluster are more similar to those in the T cell cluster than to those in their own cluster. They may have been assigned the wrong cell type or might be doublets. Thus, the Sleepwalk exploration can alert the analyst to the need for further investigation of possibly misleading features of a dimension-reduced embedding.

In Figure 2, A through C, the color scale was left at the automatically chosen range of only very small distances. When switching the color scale to a wider distance range, we can also see here how relationships between clusters (Fig. 2D) appear in the supplied distance values: We see which clusters are more and which are less similar to each other—information that a static t-SNE does not show, due to the method's design focus on faithful representation only of neighborhoods. Care is needed here, however: Once the considered distances exceed what one might consider as “close neighborhood,” the choice of distance metric used will strongly influence interpretability of the visualization, as we discuss below.

Feature space distances

The colors in Sleepwalk are meant to indicate similarity or dissimilarity between the cells’ expression profiles, quantified as distances. There are multiple suggestions for useful distance measures in the literature, and the users can provide whichever they prefer. To produce the t-SNE embedding in Figure 1, we followed the Seurat workflow (Satija Lab 2018), which calculates distances in a specific manner (Methods), and these are then also used by the t-SNE routine. We have also used these same distances to color the points in the Sleepwalk rendition (Fig. 2), thus allowing us to see directly where t-SNE succeeded and where it failed in its design goal of preserving the neighborhood relation in its input data.

t-SNE uses a flexible approach to define the distance scale over which cells are considered neighbors: It adjusts the distance scale for each cell such that all cells have approximately the same number of neighbors (the so-called perplexity). Sleepwalk, in contrast, uses a fixed distance scale. This is on purpose: It allows us to note where the neighborhood has longer or shorter range (as shown in the comparison of Fig. 2, A and B). The app offers two buttons to increase or decrease the scale of the distance-to-color mapping, allowing the user to manually choose what distance should be considered as close.

Comparing embeddings

With the availability of choice in dimension-reduction methods, the question arises of which one to use. Benchmark comparisons may address this question in general; see, for example, Becht et al. (2019) for a comparison of UMAP with t-SNE and related methods. When working on a specific data set, however, simply calculating multiple embeddings and comparing them side by side might be even more helpful. We demonstrate this here using data from a study of the development of the mouse cerebellum (Carter et al. 2018). In Figure 3, we show cells from development time point E13.5, first visualized with t-SNE (Fig. 3A), then with UMAP (Fig. 3B). The live apps in the Supplemental HTML show, in addition to this, the same comparison for the cord-blood data of Figures 1 and 2.

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 3.

Sleepwalk being used to compare two embeddings of the same single-cell data of a developing murine cerebellum at embryonal time point E13.5 (Carter et al. 2018): t-SNE (A) and UMAP (B). The user can explore one embedding in the same way as in Figure 2, while all other embeddings that are displayed concurrently are “slaved” to the one under the mouse cursor: Each cell has the same color in all embeddings. The red arrow shows the current mouse position. A live version (for these data and for the cord-blood data) can be found in the Supplemental HTML file.

To compare the two embeddings, we need, at minimum, a way to see which points in the two plots correspond to the same cells. A classical approach is “brushing” (Becker and Cleveland 1987): Selecting with the mouse, a group of adjacently depicted cells in one plot causes them to be highlighted in the other one, too. Sleepwalk adapts this idea, but instead of the usual brush, we simply use in all embeddings the same color for points corresponding to the same cell. Moving the mouse over points in one plot then highlights the neighborhood structure induced by the feature space distance chosen for that embedding not only there but also in all displayed embeddings and so links them. This allows us to see, for a structure in one embedding, whether there are corresponding structures in the other embeddings.

In the example shown in Figure 3, we see a clear correspondence between the major structures generated by t-SNE and by UMAP. Even the arrangement of cells within these structures is the same, which one can follow in the live version of the app. There are, however, also differences: The cells at the mouse position in Figure 3 are part of the connecting “filament” in the t-SNE embedding but lie in an external “protrusion” in the UMAP. Further exploration in the live version of Figure 3 can suggest that UMAP forced the two branches to intersect while still trying to repel cells of different lineages away from each other (note the gap between cells in the highlighted branch in Fig. 3B). This is another example of the dimensionality reduction artifacts that are hard to notice from a static image but can be uncovered with Sleepwalk.

Comparing samples

Until recently, most single-cell RNA-seq studies analyzed only a single sample comprising many cells. Yet, the full value of the technique might become apparent only when it is used to compare many samples. One currently popular approach to do so visually is to simply combine the data from the cells of all samples into one large expression matrix and perform t-SNE or UMAP on this. Often, global differences between samples, typically due to technical effects (Tung et al. 2017), will prevent similar cells from different samples from appearing in the same cluster or structure in the dimension-reduced embedding. Methods to automatically remove such sample-to-sample differences (e.g., the CCA-based method in Butler et al. 2018 and the MNN method in Haghverdi et al. 2018) address this issue but will not work always and may risk also removing the biological signal.

A visual alternative is to produce a dimension-reduced embedding separately for each sample and then try to find correspondences between the features in these. In Figure 4, we show how Sleepwalk allows one to perform such an exploration comparing UMAP renderings for the two E13.5 and one of the E14.5 samples of the mouse cerebellum data set. Exploring the data with the mouse shows the two replicas of E13.5 samples (Fig. 4A,B) are almost identical. The two branches (GABAergic and glutamatergic neurons) can be easily followed from the early progenitor cells up to the most differentiated ones. Comparing between the two E13.5 replicates reveals which aspects of the peculiar two-pronged shape of the glutamatergic branch are simply due to random variation and what seems reproducible. In the later E14.5 sample (Fig. 4C), the branches have disconnected from the progenitor cells, but Sleepwalk allows us to still identify corresponding cells. Sleepwalk can show that the GABAergic lineage is differentiated further in the E14.5 than in E13.5 samples, as the endpoint of the branch in E14.5 corresponds to an intermediate point in E13.5. Sleepwalk allows one to discover such details immediately, with minimal effort. Of course, such a visual exploration cannot replace a tailored detailed analysis, but it does provide a starting point and a first overview.

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 4.

Sleepwalk in multisample mode, comparing three samples of a developing murine cerebellum: two samples of two different mice embryos at time point E13.5 (A,B) and the third (C) from E14.5. The red arrow shows the current mouse position. The dashed gray lines roughly indicate two different lineages and their common progenitor cells (expression of the marker genes that we used to draw the boundaries is shown in Supplemental Fig. S1). By following the GABAergic branch, one can notice that its very tip in the E13.5 samples corresponds to cells in the middle of the branch in E14.5, indicating that the cells have differentiated further during the elapsed day.

Crucially, using Sleepwalk's multisample comparison mode does not require any removal of global sample-to-sample differences with batch-effect correction methods. If the user selects a cell with the mouse in one sample, the cells that are similar to it will be highlighted, both in the same sample as well as in all other samples. This works even if the cells in the other samples seem more distant, due to the additional sample-to-sample distance; we might only need to increase the scale of the distance-to-color mapping for the cross-sample comparisons.

Comparing distance metrics

In the examples discussed so far, we have always colored cells according to the default distance calculated by the Seurat workflow, namely the Euclidean distance in the space spanned by the first few principal components according to a PCA performed after certain preprocessing. The reason for this was not that this specific distance metric should be considered more correct or more “true” than any of the alternatives discussed in the literature but simply because it is the distance metric that has been used as input to t-SNE and UMAP when calculating the discussed embeddings.

While this specific distance metric is popular due to its appearing in standard workflows such as Seurat's, this is, of course, no reason to consider it as more correct or “true” than possible alternatives or modifications. For instance, we may either choose to use all genes in the distance calculation or only some genes, which may either be chosen for having high expression or high signal-to-noise ratio, or perhaps chosen, via manual curation, as especially informative with respect to cell type or state. We may use the genes as they are or aggregate them before into meta- or eigen-genes, for example, by a principal component analysis (as done in the Seurat workflow). The way the expression data have been transformed, normalized, or preprocessed can be understood as part of the choice of distance metric. The last, but not the least, choice is, of course, the metric itself. There are numerous ways to calculate distances from the selected and possibly preprocessed set of features. Besides Euclidean, one can calculate angular (“cosine”) or correlation distance or use kernel functions (Phillips and Venkatasubramanian 2011), as has been done, for instance, in Wang et al. (2017). Metrics can even be learned to suit the specific task the researcher has in mind (Yang and Jin 2006). Dimensionality reduction techniques are typically based on the assumption that, in feature space, cells are located on the surface of a smooth manifold. The methods attempt to learn the manifold and then to replace the original distance with the geodesic one (i.e., distance within the manifold) (Cayton 2005; Moon et al. 2018). Diffusion distances (Coifman and Lafon 2006) are a popular way of obtaining a manifold-following distance in a simple and efficient manner.

Some of these metrics may yield similar results; others can drastically change cell-to-cell distances. In order to account for the choice of metric, Sleepwalk offers a variant to the mode for comparing embeddings described above, in which points that correspond to the same cell will get different colors in different panels of the app, each showing the same embedding but having a different distance matrix assigned to it. By hovering the mouse over a cell, the user can see how the cell neighborhoods differ between the distance metrics.

Figure 5 shows Sleepwalk in the distance comparison mode. The live version of this figure can be found in the Supplemental Material. Once again, we use the murine cerebellum data set (specifically, stage E13.5, visualized with UMAP) as an example. As before, we used the 2131 genes chosen by the Seurat workflow as “variable,” and then calculated distance matrices using four metrics: (1) Euclidean distance based directly on the normalized and logarithmized expressions of these genes (Fig. 5A); (2) Euclidean distance in the space spanned by the first 50 principal components of a PCA performed using the variable genes (Fig. 5B); (3) diffusion distance based directly on the genes’ expression (Fig. 5C); or (4) on the first 50 principal components (Fig. 5D). As expected, Euclidean distance calculated on all variable genes (Fig. 5A) is almost useless when applied to so many dimensions. Most of the distances are condensed around some median value, making it almost impossible to distinguish any patterns in the data. However, all other distances (Fig. 5B–D) are already good enough to see the two developmental branches, with perhaps the diffusion distance separating them most clearly.

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 5.

Metric comparison using Sleepwalk. All four panels now show the same embedding: a UMAP visualization of the E13.5 sample of murine cerebellum. The cells are now colored based on four different metrics: Euclidean distance based directly on the normalized and logarithmized genes’ expressions (A); Euclidean distance in the space spanned by the first 50 principal components of a PCA performed using the variable genes (B); diffusion distance based directly on the genes’ expression (C) or on the first 50 principal components (D). The red arrow shows the current mouse position, which is on the intersection of GABAergic and glutamatergic lineages. Color scales were adjusted so that they roughly stretch along the entire selected branch. The spread of coloring onto the other developmental branch shows how good the metric is in separating the two lineages.

In the live version in the Supplemental HTML file, we also demonstrate a metric comparison using the cord-blood data set. There, the difference in the metrics does not lie in the details of its calculation but stems from using different input data, as each cell has been assessed in two modalities, and distances are calculated either from the single-cell RNA-seq or from the single-cell proteomics (CITE-Seq epitome) aspect of the data.

One should keep in mind that, in the example of Figure 5, we are comparing the four metrics not just to each other but also to the fifth one: the distance that was used to generate the embedding. UMAP is one of the manifold learning dimensionality reduction techniques, and as such it might be more similar to diffusion distances than to the Euclidean metric, even though the Euclidean distance in PCA space (distance [2]) was used as input for the UMAP process. Taking this together, one might expect the combination of diffusion distance and PCA to correspond especially well with the embedding. Which of the four metrics, however, should be considered “best” is a completely different question, as the suitability of metric will depend on the task at hand. While a metric can be effective in separating specific cell types, it might at the same time fail to arrange cells by their cell cycle stage (Buettner et al. 2015), and this can be considered a good or a bad thing, depending on whether differences due to cell cycle are considered a nuisance or a topic of interest in one's experiment. We therefore do not wish to provide opinions or guidance on this question in the present paper. What Sleepwalk does offer here is a means to explore differences between metrics and embeddings and understand them, not necessarily to perform benchmarks. Once a researcher is aware of such differences, it is up to him or her to decide if they affect data interpretation.

Beyond single-cell transcriptomics

In all the examples discussed so far, the points correspond to individual cells in samples assessed with single-cell transcriptomics. Another important use for dimension-reduction methods is for large-scale studies comprising dozens or even hundreds of samples. Brawand et al. (2011), for example, describe a collection of 131 bulk RNA-seq data sets comparing organ samples from several species and provide an overview PCA plot as their Figure 1; Dietrich et al. (2018) use a t-SNE plot to illustrate similarities and differences between their 246 blood cancer samples. Clearly, Sleepwalk can also be useful to explore dimension-reduced embeddings arising in such applications.

Research on dimension reduction originated in the machine learning field, with the original applications being the study of training data for machine learning applications. Of course, in this area, as well as in other applications of dimension reduction, Sleepwalk should also prove useful.

Usage

Sleepwalk is provided as a package for the statistical programming language R (R Core Team 2019). The main function of the package is also called “sleepwalk.” The user provides it with the 2D coordinates for each object (cell) in the embedding and a square matrix of cell-to-cell distances, or, alternative to the latter, a data matrix from which sleepwalk can calculate Euclidean or angular distances. For both these parameters, the user can also supply multiple matrices in order to display multiple embeddings concurrently for comparison. This can be done either such that each embedding represents the same objects (as in Fig. 3) or that each embedding represents a different set of objects but distances are given also between objects in different embeddings (as in Fig. 4).

Sleepwalk can easily be used in combination with other single-cell analysis frameworks. Visualizing, for example, a Seurat data object can be done with one line of code, as explained on the documentation web page.

By default, the sleepwalk function displays the visualization app in a web browser. Alternatively, it can also write it into an HTML file, which can then be opened with any web browser without the need of having R or the Sleepwalk package running or even installed. This is useful when an analyst wishes to share a Sleepwalk visualization with colleagues or provide it on a web page or in a paper supplement.

For a description of further options of the function, please see the documentation.

The app offers a “lasso” functionality: The user can encircle a group of points with the mouse, and the indices of these points are then reported back to the R session, where they can be queried with a callback function. This is helpful if the analyst spots an interesting set of cells while exploring an embedding and wishes to perform further analysis on them.

We also mention the “slw_snapshot” function, which produces static plots, like the figures in this paper.