genomic hypothesis generation

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

Tasks, Techniques, and Tools for Genomic Data Visualization

N gehlenborg.

• Author information
• Article notes
• Copyright and License information

Authors contributed equally to this work

Issue date 2019 Jun.

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Genomic data visualization is essential for interpretation and hypothesis generation as well as a valuable aid in communicating discoveries. Visual tools bridge the gap between algorithmic approaches and the cognitive skills of investigators. Addressing this need has become crucial in genomics, as biomedical research is increasingly data-driven and many studies lack well-defined hypotheses. A key challenge in data-driven research is to discover unexpected patterns and to formulate hypotheses in an unbiased manner in vast amounts of genomic and other associated data. Over the past two decades, this has driven the development of numerous data visualization techniques and tools for visualizing genomic data. Based on a comprehensive literature survey, we propose taxonomies for data, visualization, and tasks involved in genomic data visualization. Furthermore, we provide a comprehensive review of published genomic visualization tools in the context of the proposed taxonomies.

1. Introduction

A rapidly growing understanding of how the genome and epigenome of an organism control molecular function and cellular processes has revolutionized research in biology and medicine. Driven by affordable high-throughput technologies that allow scientists and clinicians to reliably obtain high quality sequence information from DNA and RNA molecules, generation and handling of genomic sequencing data are now routine aspects of many basic science and clinical research projects in biology and medicine.

While a large amount of genomic data is produced within individual small scale projects, there are also numerous national and international efforts to generate genomic data at large scale. These kinds of projects include efforts to catalog genomic features across cell types and tissues (e.g. ENCODE Consortium [ ENC12 ]), studies aimed at understanding fundamental principles of DNA architecture (e.g. 4D Nucleome Project [ DBG*17 ]), as well as disease specific studies that aim to elucidate the molecular changes that cause diseases such as cancer (e.g. The Cancer Genome Atlas [ CWC*13 ], International Cancer Genome Consortium [ The10 ]). Yet other projects aim to develop new approaches for early identification of genetic risk factors (e.g. Baby-Seq [ HACB*18 ]). Many of these projects do not only generate genomic data for many samples, but also produce dozens of different types of genomic data.

Visualization of genomic data is frequently employed in biomedical research to access knowledge within a genomic context, to communicate, and to explore datasets for hypothesis generation. As biomedical research is increasingly data-driven and many studies lack well-defined hypotheses [ Wei10 , Gol10 ], it is a key challenge to discover unexpected patterns and to formulate questions in an unbiased manner in vast amounts of genomic and other associated data.

Over the last two decades, hundreds of visualization tools for genomic data have been published. The large number of tools are an indicator for the broad application of genomic data and a sign that visualization of genomic data is a complex problem and active research domain.

Several challenges in visualizing genomic data are directly connected to how genomes are organized. Genomes are collections of one or more chromosomes, which are individual molecules that encode information as a sequence of nucleotides. Although genomic information is stored in the form of a sequence, the function of the genome is influenced by and requires various types of long-and short-range interactions between non-adjacent regions of the sequence. This includes interactions within and between chromosomes. Patterns in genomic data can be found across many scales, ranging from the size of whole chromosomes, which can span hundreds of millions of nucleotides, down to individual nucleotides. Another important aspect of many genomes is the sparse distribution of many types of patterns along the genome sequence.

Furthermore, the questions that are addressed with genomic data are aimed at the understanding of complex biological systems where all components are highly interconnected and influence each other. For example, the regulation of gene activity is controlled by the presence or absence of particular regulatory proteins, chemical modification of parts of the DNA, and the 3D structure of chromosomes, all of which are changing depending on environmental and other factors. An abundance of proteins, chemical modifications, and 3D structures can be measured comprehensively and mapped to the genome. While this is a greatly simplified view, it shows the diversity and number of data types from multiple sources that often need to be integrated into visualization in order to interpret genomic information.

The combination of long sequences, sparse distribution of patterns across multiple scales, interactions between distant parts of the sequence, and large numbers of diverse data types pose numerous visualization challenges. These require the design and development of specialized tools. Additionally, the number of features, the size of the datasets, and the diversity of data types all make tight integration of genomic data visualization tools with algorithmic tools a requirement for efficient analysis workflows. This further complicates the design of effective visualization tools for genomic data.

As the sequential organization is a key characteristic of genomic data, we limit the scope of this survey to visualizations that incorporate one or more genomic coordinate systems and present data in the order defined by the sequence of that coordinate system. This explicitly excludes many techniques that are based on reorderable matrices and node-link diagram approaches such as visualization of gene expression levels as matrix-based, clustered heatmaps or visualization of gene regulatory networks as node-link diagrams with expression data mapped overlaid onto the nodes. Since the presence of a genome sequence is required for inclusion in this survey, we also excluded tools for genome assembly, which is the process of defining the sequence of a reference genome for a given species.

Our survey makes two major contributions: In Section 4 , we propose taxonomies for data, visualization, and tasks involved in genomic data visualization. In Sections 5 and 6 , we provide a comprehensive review of published genomic visualization tools in the context of the proposed taxonomies. In addition, we discuss current challenges and research opportunities in genomic data visualization.

2. Biological Background

2.1. dna structure.

Deoxyribonucleic acid (DNA) is a molecule contained in living cells carrying the genetic instructions to build every biological function and molecule of an organism. DNA is studied for numerous reasons, including analyzing cancerous DNA to find treatments, finding possible risk factors for certain diseases, and comparing DNA of different species to study them in the context of evolution.

DNA consists of two complementary strands coiled up in a double helix. Each strand is composed of smaller units called nucleotides, each consisting of a base (either Adenine (A) , Cytosine (C) , Guanine (G) , or Thymine (T) ), a sugar, and a phosphate group. Biological information is stored in the order of the different nucleotides. The two strands, called forward and reverse strands, are connected at the bases. They are called complementary because an A in the forward strand corresponds to a T in the reverse strand as well as G corresponds to C and vice versa. Therefore, often only one of the strands is considered when analyzing or visualizing genomic data. In prokaryotes (bacteria and archaea) DNA is organized in one single circular sequence, which is called a chromosome, while eukaryotic DNA is usually organized in multiple chromosomes (multiple sequences).

A gene is a sequence of nucleotides encoding for a protein, a molecule which has a biological function in the organism such as catalyzing reactions (as an enzyme) or being a building block of a tissue. In order to build a protein using the information of a gene, the DNA has to be transcribed to mRNA and translated into a protein (see Figure 2 ). The process of transcribing genes into mRNA is called gene expression. The rate at which a gene is expressed (and translated into a protein) is not the same at all times but depends on many different regulatory factors, such as molecules called transcription factors . Certain sequences of the RNA molecule can initiate the transcription of a gene, called promotors, which are located upstream of the gene sequence. Transcription factors can bind to sequences in the promotor region to regulate gene expression (see Figure 4 ).

Genes on a DNA molecule are transcribed onto a mRNA molecule and translated into amino acids to form the final protein product. A gene consists of coding parts (exons) and non-coding parts (introns). In a process called alternative splicing exons can be combined in various ways to form different protein products.

DNA is organized in histones. When the DNA is wound up around the histones transcription cannot occur. When it is unwound transcription factors can bind and initiate gene expression. Epigenetic factors control to which extent the DNA is wound up.

In eukaryotic organisms, not the entire DNA sequence encodes for genes. Instead, the sequence consists of protein-coding parts called exons and non-protein-coding parts called introns (see Figure 2 , top). During transcription, introns are cut out and neighboring exons are combined to form genes. In a process called alternative splicing, one gene can encode for multiple different protein products, called protein isoforms by combining different exons (see Figure 2 , middle). The exons not needed for the formation of the protein product are spliced out. Knowledge about the abundance of isoforms is important for biologists to understand both normal processes and diseases in order to eventually improve treatment through targeted therapies.

Every triplet of nucleotides of the mRNA molecule encodes for one amino acid, the basic building block of a protein. Each triplet is called a codon . There are 64 different codons encoding for 20 amino acids and three codons signaling a stop of translation.

2.2. Mutations

When a cell divides, each daughter cell receives a copy of the cell’s DNA. This process requires copying the DNA, which can lead to errors. Moreover, DNA can be damaged by external factors such as radiation or carcinogens. Errors can be divided into small-scale and large-scale mutations. Small scale mutations include the insertion or deletion of one or multiple nucleotides and substitutions of single nucleotides. Substitution mutations can alter an amino acid in the resulting protein or a premature termination of transcription. Insertions and deletions can change all the triplets succeeding the mutation, which often leads to a completely altered protein product.

Large-scale mutations include amplifications and deletions of entire regions on a chromosome, which can lead to an increased dosage of genes in these regions or the loss of genes called copy number variation . Furthermore, parts of separate genes can be fused together to form a fusion gene. Another type of large-scale mutations is chromosomal rearrangement. For example, parts of DNA can be exchanged between non-homologous chromosomes or the orientation of chromosomal segments can be inverted.

2.3. Sequencing

Sequencing is the process of determining the sequence of nucleotides of a DNA or RNA molecule. With most of the current techniques it is not possible to sequence an entire genome at once, but the sequence has to be broken down into little pieces which are sequenced separately called sequencing reads. To reconstruct the entire genome, DNA sequencing reads have to be puzzled together by using overlaps at the end of the reads and often an already sequenced genome (called a reference genome). The process of reconstructing the sequence from sequencing reads is called assembly .

DNA sequencing data is then further analyzed to find mutations or structural rearrangements or for the comparisons to other species or individuals. RNA sequencing data is usually not assembled but used to determine expression levels or patterns of alternative splicing by mapping the RNA sequencing reads to the DNA sequence.

2.4. Alignment

Sequence alignment is one of the most important operations performed in the analysis of genetic information. It is often used to find functional or evolutionary relationships between sequences stemming from different individuals/species.

In order to align multiple sequences, they have to be arranged in a way that makes it possible to identify similar regions. A good alignment maximizes the number of shared symbols in one column while minimizing gaps and non-matching symbols (mismatches) and retaining the sequence order. Figure 3 shows a simple example of a multiple sequence alignment (MSA) .

A multiple sequence alignment (MSA) of the sequences ‘ACGTCATCA’, ‘TAGTGTCA’ and ‘CGTCATA’.

2.5. Epigenetics

Monozygotic twins are genetically identical. However, especially older twins often show significant differences in their appearance and they sometimes even have acquired individual diseases. While not all factors for this phenomenon are understood, epigenetic differences have been identified as correlating with different phenotypes in twins [ HGW09 , FBP*05 ]. Through epigenetic processes, genes can be turned on and off without altering the genetic sequence, often influenced by environmental factors or stochastic processes.

In order to compact and organize chromosomes, eukaryotic DNA is wound up around proteins called histones. Compacted DNA cannot be transcribed since it is not accessible for transcription factors (See Figure 4 ). Epigenetic processes such as DNA methylation or histone modification can control the extent to which DNA is wound up and unwind parts of it to make it accessible to transcription factors. Therefore, expression and protein synthesis can be controlled without changing the underlying DNA sequence. These changes can be inherited by daughter cells, as well as by the descendants of the organism. Since the described changes do not affect the DNA sequence, they are called epigenetic modifications (from Greek epi meaning “over, outside of, around”).

2.6. Chromosome Conformation Capture

Another aspect to consider when studying different phenotypes is the 3D structure of DNA. Nucleotides separated by many positions in the sequence can be in very close proximity in 3D space, as indicated by Figure 4 . Nucleotides can be close in 3D space when they are wound up around the same histone, or when they are part of a loop that controls transcription.

With chromosome confirmation capture techniques, the interactions of genomic loci in 3D space can be quantified. One way of doing this is measuring the interactions between fragments of the genome. With the Hi-C technique the interactions between all possible non-overlapping fragments of a genome can be determined resulting in a matrix of interaction frequencies (also called contact frequencies). The higher the interaction frequency of two fragments, the smaller the distance between them in 3D space.

2.7. Genome Evolution

Environmental pressures during the evolution of species lead to changes in sequence and composition of the species’ genomes. Not every region of the genome changes at the same rate. Sequences that have a function in fundamental processes (genes, exons) are more similar or more conserved across different species. Moreover, the order, orientation and location of subsequences can change over time. A gene shared between two species can be at different chromosomes in a different genomic neighborhood. Genomic synteny refers to the order of conserved blocks within two sets of chromosomes that are being compared. Two sequences are called syntenic if they contain similar blocks of genes in the same relative positions in the compared genomes.

2.8. Previous Literature Surveys

The following literature surveys focusing on genomic data visualization are all aimed at audiences in the bioinformatics and biology research communities. Therefore, their review of visualization tools is often more focused on features available to the users than on formalizing the description of common tasks and techniques.

Nielsen et al. [ NCD*10 ] reviews the techniques and challenges in visualizing genomes with a focus on three core user tasks: (i) analyzing sequence data, both in the context of de novo assembly and of re-sequencing experiments, (ii) browsing annotations and experimental data mapped to a reference genome and, finally, (iii) comparing sequences from different organisms or individuals. They review several stand-alone and web-based tools and compare their cost, operating systems, and compatibility. Despite technical advancement, several challenges for analysis and visualization of genomics data remain due to the volume and heterogeneity of these data. The authors also recommend ways of improving the design of these visualization tools. For example, a high-level overview of data, or providing recommendations for where to look at, can improve user efficiency. In addition, Nielsen et al. suggest that new genome browsers should build on the successes of earlier tools, allowing easy cross-platform access, customization of data and display, and the ability to perform on-the-fly computation within the visualization. The authors point out that although several successful visualization tools are used for specialized analysis demands of the users, there is a great need to improve the integration among tools and ease the transition from one analysis to another.

Schroeder et al. [ SGPLB13 ] review common visualization techniques for exploring oncogenomics data and compare several existing tools. They describe genomic coordinates that help researchers find answers to questions about alterations tied to genomic loci, or to inspect some particular genomic locus. Heatmaps are frequently used to describe transcriptomics and genomics data stored in the form of matrices. Node-link diagrams are used to visualize functional relationships between different entities, such as genes. Qu et al. [ QLN*19 ] also review visualization methods for oncogenomic data, such as scatterplots, networks, heatmaps, clusters and the combination of machine learning and visualization. Moreover, they discuss future trends in this field. Pavlopoulos et al. [ PMP*15 ] conducted a comprehensive review of general genome visualization tools and summarize them into four categories: genome alignment visualization tools, genome assembly visualization tools, genome browsers, and tools to directly compare different genomes with each other for efficient detection of genomic variants.

Yardimci et al. [ YN17 ] reviewed five visualization tools for genomic interaction data generated using chromosome conformation capture approaches. They characterized the visualization functionality of those tools based on available visualization types and also discussed integration of supplementary views and data handling capabilities. They categorized visualization techniques based on whether they are more suitable for short-range interactions or long-range interactions. Goodstadt et al. [ GMR17 ] reviewed visualization challenges for 3D genome architecture and provide a taxonomy of tasks outlining essential features of 3D genome visualization. These tasks and challenges including data representation, data refining, and data interaction.

O’Donoghue et al. [ OBC*18 ] surveyed how visualization is being used in a broad range of data-driven biomedical research areas. The authors reported on current visualization techniques and challenges in genomics and epigenetics, RNA biology, protein structures, systems biology, cellular and tissue imaging, and populations and ecosystems. They also pointed out the limitations in popular tools such as the widespread use of rainbow color maps and recommended that tailored visualization methods and tools are necessary for advancements in biomedical insights.

Unlike the surveys mentioned in the previous section, this survey is aimed at the visualization community and bioinformatics researchers who develop visualization tools and focuses on visualization tasks, techniques, and tools. A secondary goal of this survey is to take a step toward bridging the gap between research in the visualization and bioinformatics communities and to highlight the promising research areas in this emerging cross-disciplinary field.

We searched both the PubMed database ( https://pubmed.gov ) and Google Scholar ( https://scholar.google.com ) for tools related to genomic visualization using the following keywords: “visualization of genomic data”, “genome data visualization”, “genomic sequence visualization”, and “transcriptome data visualization”. These searches resulted in a seed collection that we considered relevant for this survey based on their titles and abstracts. To this seed collection, we also added paper describing tools and methods that we were familiar with but had not been returned by the search.

Using this seed collection, we identified several tool categories such as genome browsers, multiple sequence alignment tools, and others, that we used for more focused searches to expand our collection. In the next step, we removed manuscripts that did not fit our scope of sequence-based visualization, resulting in a total of 111 papers. We further removed papers that only marginally mentioned visualization or did not present a tool or technique. We also removed papers that described re-implementations of a particular technique, such as Circos or multiple sequence alignments, if there was no novel aspect to the visualization. In such cases, we focused on those papers with a higher citation count.

Ultimately, the collection surveyed for this review contains 83 papers describing genomic visualization techniques or tools. Of the 83 surveyed papers, 7 papers (8%) were published in visualization venues (all IEEE TVCG) and 76 manuscripts (92%) were published in bioinformatics and biology venues.

4. Taxonomy

In this section, we will provide an abstract understanding of genomic data and the basic parameters and techniques for its visualization. We introduce three taxonomies: A data taxonomy characterizing genomic features, a task taxonomy categorizing the most important tasks for genomic visualizations and a visualization taxonomy which we use to categorize tools.

4.1. Genomic Features

4.1.1. types of features.

A genomic feature is a data point of measured or knowledge-based data that can be mapped to genomic coordinates and has an extent of one or more nucleotides. Knowledge-based data represents the knowledge we have about a genome without conducting new experiments, which includes a reference genome with a known sequence and annotations. For example, gene annotations and functional annotations. The reference genome represents the known sequence of a genome of a species. It is not the genomic sequence of one individual, but it is derived from a group of individuals. An analogy for knowledge-based data can be found in the visualization of geological maps. Usually, geological structures such as mountains and rivers are named or elevations are indicated by numbers, which can be understood similarly to the annotation of a reference genome. In the context of maps, measured data can be for example, traffic data, the size of cities, population data or election results. For genomic data, measured data is anything that can be measured about a genomic sequence or is derived from that measurement. Examples of data that can be measured include the sequence of DNA in a sample, epigenetic signals, and contact frequencies. Derived data is often created by setting measured data in context with the knowledge based data. For example, by comparing sequencing data of a cancerous sample to a reference genome, mutations and genomic rearrangements can be deducted.

Depending on properties of the underlying data, features can be of different types. Features that only cover one nucleotide are defined as point features , while features covering more than one nucleotide are segment features . Features can be associated with zero, one or multiple attributes which can be quantitative, ordinal or categorical. A feature with zero attributes only shows position and extent, such as the position and extent of a gene. If the gene is associated with other data, such as names, functions and expression levels, the feature is associated with multiple attributes.

4.1.2. Feature Sets

Data taxonomy, visualization taxonomy, and task taxonomy for genomic visualizations. The data taxonomy describes how different genomic data types can be encoded as feature sets . A genomic visualization contains one or multiple coordinate systems applying a specific layout , partition , abstraction and arrangement (in case of multiple axes) of sequence coordinates. Feature sets are encoded as tracks and placed on the coordinate systems. Multiple tracks can either be aligned by stacking or overlaying. A visualization can consist of one or multiple views , each containing a set of aligned tracks. Multiple views can show data on one or multiple scales and foci . The task taxonomy explains how different search and query tasks operate on genomic visualizations.

Feature sets can be combined by intersecting or uniting them. For example, consider the combination of a set of sparse features, such as coding region, with a set of contiguous features encoding for an epigenetic signal across the sequence. Features in the resulting set are associated with two attributes: the epigenetic signal and a Boolean attribute that encodes whether they are in a coding region.

4.1.3. Meta data

Feature sets are usually associated with meta data. For example, a feature set containing expression data of a patient sample can be associated with data about the sample itself and the sample donor, such as the date when the sample was taken, the type of tissue, if the donor has cancer or if he is a smoker. Similarly, in our map example, it could be the date when the map was created or the type of the map. Meta data helps put the data into context. For example, it can help identify possible correlations between different phenotypes and measured data.

4.2. Visualization

4.2.1. sequence coordinate systems.

Theoretically, a sequence can be visualized in any layout preserving the sequential nature. In practice, most genomic visualizations display sequential data either in a linear or a circular fashion (see Figure 5a ). The genomic coordinates of a sequence correspond to a sequence axis, which is visualized in a layout. A sequence axis is a coordinate system for genomic features. (see Figure 5d ).

Layout, abstraction, partition and arrangement of sequence axes. (a) A sequence axis can be displayed in a linear, circular, space-filling or spatial layout. (b) A sequence axis can be visualized completely or some parts or the entire sequence can be abstracted. (c) Distinct sequence parts can be visualized in as a whole (contiguous) or segregated. (d) Two sequence axes can be arranged in different ways.

Abstraction

Arrangement

4.2.2. Genomic Tracks and Matrices

For genomic data it is important to put multiple different data types into context in order to draw conclusions. For example, when analyzing mutations in a genome it is useful to visualize them together with gene annotations to estimate their functional impact. In order to analyze multiple data types at the same time, a genomic visualization often contains multiple tracks . A track is a visual representation of genomic data with one or multiple parallel sequence axes showing one feature set. Typically, tracks are oriented horizontally, but in some tools they can also be oriented vertically. The annotations and mutations in the stated example are two features sets that are represented by two separate tracks.

Track Types

For each type of feature sets a separate track type can be defined as proposed by Gundersen et al. [ GKA*11 ]. According to the authors seven different basic track types can be defined (excluding interconnections). For sparse feature sets they developed four track types. A track for a sparse feature set with only features of length 1 with zero attributes is called a point track . For example, the positions of all substitution mutations could be displayed with this track type. When the features in a point track are associated with an attribute, such as the substituted nucleotide, the track corresponds to a valued point track . Respectively, segment tracks can encode for the position and extent of genes, while valued segment tracks can additionally show an attribute, such as the gene expression or gene name. For contiguous feature sets the track types correspond to ungapped versions of the previously described types. Contiguous feature sets containing features with a length produce a genome partition track . If the features are associated with an attribute, the track corresponds to a step function . A valued point track without gaps corresponds to a function .

Gundersen et al. [ GKA*11 ] also propose eight extended track types which can additionally encode for interconnection features. Seven correspond to the previously described types, but pairs of features are associated with interconnections which can be directed and/or have a weight. In scope of this review paper we allow the association of multiple attributes or complex attributes to track elements and interconnections. For example, a track should be able to encode for the distribution of different nucleotides at point mutations, which corresponds to a valued point with a complex attribute.

Theoretically, all of the described feature set types can be encoded with these track types. However, especially Hi-C data is usually visualized in arrangements using more than one sequence axis. In Hi-C data all pairs of contiguous segments of a specific size in the genome (bins) are associated with an interaction frequency, which represents an undirected weighted interconnection. Since this kind of data is usually easier to display using two axes we distinguish tracks showing features on a single axis (one-axis tracks) and tracks showing data on two axes (two-axes tracks). A matrix is a special form of a two-axis track showing data on two sequence axes that are arranged orthogonally.

Visual Encoding and Track Alignment

Some common visual encodings of tracks and matrices can be seen in Figure 6 . One of the most commonly used encodings is color. For example, valued point tracks showing variants can be encoded by coloring corresponding to the variant type, valued segments showing genes can be colored by functional category or gene expression (continuous color scale).

Examples of visual encodings of feature sets and arrangement of tracks. Features can be encoded through color, height of blocks and positions (a,b,c,d). A two-axes track can be arranged with multiple one-axes tracks (e,f).

A sparse set of segment features can contain segments that can overlap. Read data is an example of this type of feature. Sequencing reads can be mapped to long sequences and usually overlap, yet they do not always cover the entire genome. Overlapping segment features can be stacked in a way that avoids visual overlaps without introducing unnecessary white space.

Categorical attributes are often also represented with symbols, such as the encoding of substitution mutations with the letter of the altered nucleotide or symbols for deletions and insertions (see Figure 6a , top). Often, nucleotides are encoded using both color and symbol. Features with continuous attributes can also be encoded using heights. Segments can be displayed as blocks with varying heights depending on the attribute value (see Figure 6b ). The function track type is usually encoded by using a line chart which spans the entire sequence.

As previously described, matrices can be used to display interconnection features. Figure 6c shows a continuous attribute of an interconnection feature encoded using a heatmap. The colors encode the attribute value, i.e. the weight of the interconnection. In contrast, the matrix in Figure 6e only shows which features are interconnected.

Undirected interconnection features (such as Hi-C data) can also be encoded on a one-axis track. For sets of undirected interconnection features, a matrix representation is symmetrical. Therefore, the matrix can be cut in half along the diagonal, rotated, and mapped to sequence coordinates forming a track. The value of an interconnection can be retrieved by following imaginary lines originating at two features in 45-degree angles. Like matrices, this type of track encoding can show interconnection without a continuous attribute as seen in Figure 6d .

Aligning multiple space-filling tracks is rather uncommon and limited. Since space-filling layouts use space most effectively, it is hard to arrange tracks in parallel. Moreover, a parallel arrangement complicates the identification of the same coordinate across tracks. For this reason overlaid track alignments are more common. In order to avoid hiding features transparent colors can be used. Another possibility is combining feature sets instead of overlaying two tracks.

4.2.3. Multiple Sequences

In the previous sections, we described how features on a single sequence can be visualized using one or multiple sequence axes. Yet in the field of comparative genomics , multiple sequences or reference genomes are analyzed to study genome evolution. Two or more sequences are compared to find blocks of high similarity on the sequence level (conservation) and to analyze if the location, order, proximity, and orientation of these blocks is similar in the compared genomes (synteny). The goal of the visual encoding of sequence comparisons is connecting sequence coordinate systems to show regions of high similarity of the sequences. Similar regions can be visualized using different techniques and visual encodings. The size of sequences to be compared can vary greatly from small regions, such as genes to entire chromosomes or entire genomes.

There are three basic techniques for visualizing sequence comparisons usually applied for genomic visualization: (i) comparison by alignment, (ii) comparison by connecting conserved blocks, and (iii) comparison by using dot plots. Especially for the visualization of many small sequences, alignment-based techniques are used, with which shared nucleotides are algorithmically aligned as described in Section 2.4 . This represents a construction of a shared coordinate system and sequence axis. Each sequence corresponds to a valued segment or point track that is aligned to the coordinate system.

Another way of visualizing comparisons is to keep the different coordinates for each sequence separate and visualize the comparison in one or multiple two-axes tracks. Interconnection features between segments in two sequences can encode for the positions and extents of syntenic regions, which can be encoded by connecting them with lines or bands or by applying the same color. Meyer et al. [ MMP09 ] describe which combination of encoding, layout and arrangement is most effective depending on the length of the displayed sequence. They recommend using bands and parallel linear or serial circular arrangements for shorter sequences (up to chromosomes) to avoid too many crossing lines and color encoding for whole genome comparisons in parallel arrangements.

Orthogonal arrangements of sequence axes can show the similarity between every position or bin of positions of one sequence to every position or bin of positions in the other sequence. The two sequences are arranged in a 90 degree angle spanning a comparison matrix. If nucleotides or bins match between positions of the two genomes, a dot is drawn in the corresponding cell. Similar regions form diagonal lines of dots. With this technique insertions, deletions and inversions can be identified.

4.2.4. View Configurations for Genomic Visualizations

In order to categorize tools and techniques, we define three parameters of a genomic visualization (see Figure 8 ): (i) the number of views that show data mapped to genomic coordinates, (ii) the number of scales used to analyze the data at the same time and (iii) the number of foci, i.e. non-adjacent genomic segments that can be viewed independently.

The number of views, scales and foci are important parameters of a genomic visualization. The combination of these parameters results in five different view configurations.

We restrict a view to a set of one or multiple aligned tracks, which contains at most one two-axis-track. A visualization can consist of one or multiple views displaying features mapped to genomic coordinates, which can be linked or independent.

A genome can be very large and analyzing it on different scales, such as the whole genome or single genes, can be of great value. Similar to a map: A view of the entire world provides a useful overview, but we cannot analyze the street structure of New York. While some genome visualizations only allow visualizing sequences on one scale at a time, others provide multiple views to visualize the data on different scales. In our taxonomy, a multi-scale visualization visualizes the data on multiple scales at the same time in multiple different views. However, also single-scale visualizations can provide a zooming interaction.

A focus can be understood as a sliding window across the genome. Only the region in this window can be analyzed. Multiple foci enable users to look at distinct segments of a genome in parallel and compare features which are dispersed across the genome. In the map analogy this would be, for example, comparing the street structure of New York to that of London with Google Maps. If we zoom out of the map, we can only see the names of the two cities; if we zoom in, we can only see one city at a time. We need to open a second browser window to view them in detail at the same time. Note that foci can be defined in a flexible way and are visualized in separate views, while sequence abstraction is usually done in a previous step, where in the case of filtering non-adjacent sequences are “glued” together.

Sometimes, arrangements of two sequence axes corresponding to the same sequence are used to facilitate the visualization of interconnection features. For arrangements of multiple axes, we have to add another dimension for scale and focus. Figure 7 shows an example of foci in an arrangement with two axes. Two different foci are shown in the tracks aligned to the linear axes. However, the matrix spanned by the orthogonal arrangement can only visualize one focus of two-dimensional data. Similarly, two orthogonal axes can be on different scales, leading to two one-axis scales and one two-axes scale.

A visualization with multiple axes can have foci in multiple dimensions. This orthogonal arrangement of axes contains two one-axis foci and one two-axes focus.

Combining these parameters in all possible ways results in five basic view configurations of genomic visualizations as seen in Figure 8 . Note that if a visualization contains two axes, foci and scale refer to two-axes foci and two-axes scales.

As we restrict a view to a set of one or multiple aligned tracks, with at most one two-axis track, it is not possible to visualize multiple scales and foci (of the highest dimension) in one view with our taxonomy. Therefore, the left branch only consists of one path. The other view configurations are combinations of multiple views with one or more scales and foci. We categorize our tools in Sections 5 and 6 using these five basic view configurations. For visualizations with multiple axes we additionally show the number of one-axis foci and one-axes scales.

4.2.5. Linking Views

Genomic visualizations often incorporate multiple views that can be (i) independent , (ii) weakly linked , (iii) medium linked or (iv) strongly linked . While independent views are not connected in any way, weakly linked views are linked by brushing and linking. Medium linked views share navigation, for example two views are at different scales, but zooming always affects both. Strongly linked views share genomic coordinates at one axis and can also share tracks that are aligned to the axis.

Utility views provide information about (i) tracks, (ii) features, or (iii) genomic coordinates. The property that distinguishes these views from e.g. detail views is that they never show genomic coordinates directly but only meta data or derived data. These types of visualizations can be either aligned with the genomic visualization, or weakly linked. As an aligned visualization, consider a view showing meta data about tracks which is situated on the left side of each track. In case of sequencing data or expression data of individuals, this could be, for example, phenotypic information about the sample donor. Since this kind of visualization is in direct association with the genomic data we call it a strongly connected utility visualization . On the other hand, consider a visualization that is connected with a table through brushing and linking. Only by clicking on an element in the visualization is the corresponding element in the table highlighted and vice versa. Since the views are not aligned we call this type of visualization a weakly connected utility visualization .

Visualization tasks represent actions that users may perform on their data [ BM13 ]. These can be both low-level operations or high-level user intents while interacting with a system. Visualization tasks have been defined and classified, often depending on the context and scope of the tasks. A common feature of most genomic coordinate visualizations is that they visualize one or multiple types of features at their corresponding positions, therefore the tasks that different tools and techniques help to solve are often similar. In this section we describe the most common tasks performed using genomic visualizations.

A typology of abstract visualization tasks proposed by Brehmer and Munzner [ BM13 ] focuses on three questions: why is a task performed, what are the inputs and outputs, and how is the task performed. What is particularly useful in this typology is that it distinguishes between user intents (that answer why) and interactions (that answer how) and provides a link between the two questions. In this section, we summarize and categorize the “Why” task for genomic visualizations. Moreover, we give an overview of common interactions (“How”) and inputs and outputs (“What”).

4.3.1. Why?

Categorization of “Why” Tasks

4.3.2. How?

“How” refers to the methods with which the “why” tasks can be solved using interaction. Independently of the described view configurations, tools for the visualization of genomic data often differ in their level of interactivity. Many tools can only plot data as a static image. Different datasets, different visual representations, regions and zoom levels can often be chosen as parameters for the plot, yet there is no interaction with the visualization itself.

4.3.3. What?

4.3.4. High-level vs. Low-level Tasks

5. Single Genomic Coordinate System

5.1. genome-scale visualizations.

Genome-scale visualizations display one or multiple regions of a genome on absolute coordinates. We further categorize genome-scale visualizations based on the type of the features that they are specialized on. They can be focused on displaying (i) non-interconnected feature sets, (ii) sparsely interconnected feature sets and (iii) densely interconnected feature sets.

5.1.1. Non-Interconnected Feature Sets

Often non-interconnected feature sets are visualized using tools that consist of one or multiple parallel tracks and visualize many kinds of different features. They display features using linear, circular and space-filling layouts (see Table 2 ). While certain data types are very common, some tools are more specialized on the visualization of a specific type of genomic data, such as the Savant Genome Browser 2 [ FSB*12 ], which is specialized on showing structural variation or HilbertVis [ And09 ] and HilbertCurve [ GES16 ], which are especially useful to display epigenetic data. Tools in this section are specialized on visualizing many sets of non-interconnected features, yet they can sometimes visualize local interconnections.

Linear Layout

A tool group known as “genome browsers” usually displays multiple parallel tracks in a linear layout. A genome browser commonly consists of three components: A reference genome, annotations, and one or multiple tracks; see Figure 9 from the Integrative Genomics Viewer (IGV) [ TRM13 ]. Genome browsers are used for exploring a reference genome together with other knowledge-based data or for comparing experimental results, such as sequencing data, to the reference genome.

The Integrative Genomics Viewer (IGV) is an example of a genome browser that can show multiple scales and foci in separate views. Moreover, the visualization includes a strongly linked utility view in form of a column next to the tracks showing meta data. Figure from [ TRM13 ].

Genome browsers usually enable the visualization of a small window of the genome and allow navigation such as zooming and panning. Most of them are not suited for meaningful overview visualizations of whole genomes, since the data must be extremely aggregated to fit on the screen. Most genome browsers are limited to the visualization of single chromosomes [ KSF*02 , NCB17 ]. Some browsers apply a predefined minimum zoom level [ DPH11 ], others show empty tracks for certain feature sets if the visualization is zoomed out too much [ TRM13 ]. In terms of the defined visualization parameters, genome browsers often consist of a single view and can therefore only visualize the data at one scale and focus at a time (1-1-1 view configuration, see Table 2 ). Consequently, features can only be explored, compared, and summarized in a relatively small window which complicates, for example, analyzing gene co-expression in non-adjacent regions. To counter this, some tools provide methods of data abstraction, with which introns can be filtered out or specific regions of interest can be placed next to each other [ KSF*02 , ZMX*11 ]. However, the regions are still visualized in the same view and to change the borders of the regions they have to be redefined.

The Savant Genome Browser [ FSB*12 ] is the only feature viewer that was grouped into the multiple views, single scale, and single focus (n-1–1 view configuration, see Table 2 ). It consists of two visualizations both showing features on genomic coordinates: a classic genome browser view and an additional view which can display population data with different visualizations. For example, it can display a heatmap which shows the patterns of alterations of single bases in multiple samples. Columns correspond to the positions of the alterations and rows correspond to samples and cells show the type of alteration. Although the heatmap shows the data on a different scale, we associated it with a single scale configuration, since it does not display the same data (the same tracks) on a different scale but constitutes an entirely different visualization.

Another category of genome browsers are “overview-detail” browsers [ Don09 , HBB*02 , FNL16 ] (n-n-1 view configuration, see Table 2 ). Like single view browsers they show one region of the genome (which means that they have one focus) but also have an additional detailed view for a part of this region. Navigation is usually linked in these browsers to ensure that the detailed view is always part of the chosen region. An advantage of this genome browser layout is that features can be analyzed in a small window as well as in their global neighborhood simultaneously.

Few genome browsers provide a very high level of flexibility by enabling the visualization of multiple focus regions on different scales [ NCB17 , KWZ*12 , TRM13 , CZ13 ] (n-n-n view configuration, see Table 2 ). Initially, they visualize single regions, but multiple regions can be selected or added through different mechanisms. IGV allows selecting a region with a user dialog [ TRM13 ], while the NCBI Sequence Viewer enables the creation of new views by highlighting a region and selecting “create new panel” in a context menu [ NCB17 ]. Regions can be selected with different extents, yet they receive the same amount of screen space. Therefore, the two region views can show features on different scales. Region views can be either arranged horizontally (IGV) [ TRM13 ] or vertically (NCBI Sequence Viewer) [ NCB17 ]. While a horizontal arrangement facilitates comparing tracks, the available horizontal space for each region is smaller. The increased flexibility of genome browsers in an n-n-n view configuration often goes along with a more complicated user interface and more possible interactions. For problems that do not require viewing different scales and regions at the same time, a genome browser in a less flexible configuration can facilitate the exploration process.

Often genome browsers contain utility views that are strongly connected to the tracks and show different kinds of meta data. When the different tracks correspond to data obtained by analyzing different samples, for example with IGV [ TRM13 ], they can display data about the sample donor, such as gender, race, ethnicity and many more.

Xena [ GCB*18 ] is another type of visualization for non-interconnected feature sets, which can be seen as a “population browser.” Unlike a genome browser, it enables the exploration of entire patient cohorts. Xena displays population data in a column-based layout, where each column corresponds to a view showing a different type of feature. Sample data is displayed as parallel tracks in each column. Tracks are sorted by columns, while the order is preserved across columns. Columns can either show meta data about the samples or genomic data, such as gene expression, copy number alterations, or somatic mutations for small regions such as genes. Columns can be added, sorted, and rearranged interactively. In contrast to genome browsers, the focus of Xena is not on visualizing data on genomic coordinates, but on stratifying and characterizing populations. Therefore, what is considered meta data and utility views in our taxonomy, can be seen as main data types and main views in Xena.

Circular Layout

Circular layouts are often used for the visualization of non-interconnected feature sets. These visualizations are commonly known as “genome maps” [ PSESVD10 , KvEB*04 , CTB*09 , SW05 , OvHSF15 , BKS*16 , GS08 ]. In contrast to linear visualizations, circular genome maps provide an overview of the entire genome and therefore allow summarizing genomic features on a global scale. Usually they visualize prokaryotic genomes, which are circular and much smaller than eukaryotic genomes. Figure 10 shows an example of a genome map created with CiVi [ OvHSF15 ]. Despite smaller genomes, data still has to be aggregated to be visualized. Even the smallest known prokaryotic genome is larger than 500 kilo bases [ FGW*95 ]. Contiguous feature sets with quantitative attributes are averaged for windows of equal size, and only segments bigger than a minimum size are displayed. For a more detailed visualization, many tools provide the possibility to visualize the entire genome [ CTB*09 ] or a small region [ KvEB*04 , PSESVD10 ] in a linear layout. Most Genome Maps are static plotting tools and do not provide direct interactivity [ CTB*09 ]. Others allow zooming and navigating the genome map [ SW05 , PSESVD10 ].

Circular visualization of Escherichia coli with CiVi [ OvHSF15 ]. The tracks show genes and GC content.

An example of a more interactive prokaryotic genome visualization is Island Viewer 3 [ DLS*15 ]. It can visualize one or two prokaryotic genomes in parallel in an overview-detail configuration (n-n-1). It uses a circular layout for the genome overview and a linear layout for the details. In contrast to the other described circular visualizations, Island Viewer is specialized. It focuses on a specific data type (called genomic islands) and compares different prediction methods for this data.

Space-Filling Layout

A visualization technique for chromosomeor genome-wide overviews of features, especially epigenomic marks [ KAS*11 ] is based on a genome layout that uses a space-filling curve rather than a linear or circular genome layout. A desirable property of space-filling curves, such as the Hilbert curve [ Hil35 ], is that they arrange 1D sequence information on a 2D grid so that features close to each other in 1D, are close to each other in 2D. Such plots can be created with HilbertVis [ And09 ] and the HilbertCurve [ GES16 ] R package (see Figure 11 ). Unlike linear and circular genome layouts, parallel arrangements of tracks are not feasible for space-filling curve layouts, limiting the approach to single or a limited number of overlaid tracks.

Hilbert curve visualization for an epigenetic marker created with the R package HilbertCurve described in [ GES16 ].

5.1.2. Sparsely Interconnected Feature Sets

Sparsely interconnected feature sets correspond to a set of segment features with interconnections that are sparsely distributed across the genome. Researchers often deal with this kind of data when studying structural variation or alternative splicing. Both fields of research are based on mapping reads to the genome and deducting patterns from this mapping. Patterns of alternative splicing can be found by mapping reads obtained with RNA sequencing to a DNA sequence. Based on the number of reads per exon and reads that contain sequences of multiple exons (junction reads), splicing patterns can be deducted. Genomic rearrangements are obtained by mapping DNA sequencing reads to the genome. Rearrangements can be found by analyzing read depth, paired-end reads, and reads containing sequences of distant regions (split reads). For a more exhaustive description of the biological background see section 2 . Sparsely interconnected feature sets are commonly visualized in a linear or circular layout (see Table 3 ). The main goals for visualizing these data types are exploring the data as well as comparing patterns across different samples and conditions. Especially for genomic rearrangement, both the global distribution of the arrangements, as well as patterns on the sequence level are of interest for the exploration.

Alternative splicing is usually visualized in a linear layout and is sometimes displayed as a track in a genome browser [ TRM13 ]. In principle, alternative splicing data corresponds to a set of segment features, where each segment encodes for an exon. Exons are connected with interconnection features, which show which exons are contained in the final protein product. More specifically, an interconnection shows that there exists at least one read that contains parts of both exons. Often, exon segments are associated with the number of reads mapped to the exon as well as interconnection features are associated with a quantitative attribute that represents the number junction reads.

This type of feature is commonly visualized in a “splice graph”, where exon segments are displayed as nodes and interconnection features as segments, as first proposed in 2004 [ XRL04 ]. This technique is applied by many tools including SpliceSeq [ RCK*12 ] and the R package SpliceGrapher [ RTRBH12 ]. A disadvantage of splice graphs is that they do not visually encode for the number of reads mapped to the exons and splice junctions but only use labels. Therefore, SpliceGrapher provides additional tracks for supported junctions and a view visualizing read coverage with an area chart.

A commonly used technique uniting splice graphs and quantitative data in one visualization are “Sashimi plots” [ KWS*15 ]. Figure 12 shows the splicing patterns of three samples with Sashimi plots created with IGV [ Bro ]. A Sashimi plot is displayed using an interconnected valued point track. Each nucleotide in an exon is associated with a quantitative attribute, the read depth, which is encoded using an area chart. Nucleotides that form junction sites are connected by interconnection features with quantitative attributes, encoded as weighted arcs. The view on the bottom of Figure 12 shows known isoforms using a segment track by encoding sequences contained in the isoform as blocks and spliced-out sequences as lines. In IGV, multiple Sashimi plots can be visualized in separate tracks, which allows the comparison of splicing patterns across samples or conditions. In order to summarize splicing patterns at a population level, the command line utility SplicePlot [ WNM14 ] produces averaged Sashimi plots to find splicing patterns that manifest as phenotypic differences between population groups.

Sashimi plots from IGV [ Bro ] enable users to analyze splicing patterns of different samples. The bottom view shows known isoforms, the top view shows splicing patterns of three samples. At exons read density is displayed with area charts, junction reads are displayed as arcs labeled with the number of reads.

Sashimi plots often show numerous interconnections, which complicates the identification of isoforms and comparisons of multiple plots. Vials [ SAB*16 ] was designed to address the shortcomings of Sashimi plots and to visualize and compare data from multiple samples at once, by distributing the data across multiple tracks showing junctions, abundances, and expression levels. Elements in the tracks corresponding to the same population are linked by brushing and linking. As Vials allows visualizing one gene at a time, it is a one view, one scale, one focus visualization. Yet, it applies a method of abstraction. Introns can be displayed in their full length or abstracted to equally sized gaps. In comparison to Sashimi plots, Vials provides a more abstract form of alternative splicing visualization which can make it harder for novices to interpret.

Similar to alternative splicing patterns, genomic rearrangements can be displayed as arcs on a sequence axis in a linear layout. Gremlin [ ORRL10 ] is an overview-detail visualization that can show rearrangement events at three separate scales, informing users about the context of where the breakpoints of a rearrangement occur (see Figure 13 ). It allows navigation across the genome by selecting regions of interest with a sliding window. Within these regions, single arcs corresponding to structural rearrangements can be selected and viewed in even more detail. In the complete genome view, as well as in the region of interest, tracks display the location and extent of deletions, inversions and translocations and copy number data. With multiple linked views, it supports both global trend analysis and local feature detection. As a result, this visualization enables users explore the high-level, complete genome perspective as well as low-level, structural rearrangement view.

Gremlin [ ORRL10 ] enables users explore the high-level, complete genome perspective as well as low-level, structural rearrangement view.

One of the most common ways to represent structural rearrangements spanning large genomic distances is by displaying the chromosomes in a circular layout and drawing arcs between connected regions with tools like Circos [ KSB*09 ]. Like Gremlin, Circos can encode translocations, inversions, and deletions using a combination of interconnections and segment tracks. Tracks showing other genomic features can be stacked and can apply visual encodings such as scatterplots and heatmaps. Depending on the size of the depicted region, Circos-style plots can show smaller or larger relationships between distant genomic regions. Though widely used, such depictions are limited in their ability to show data at varying scales. Zooming on a circular genomic representation is unintuitive and seldom implemented. ClicO Free Service [ CTYN15 ] is an online web-service which provides a user-friendly, interactive, web-based interface with configurable features to generate Circos circular plots.

5.1.3. Densely Interconnected Feature Sets

Densely interconnected feature sets correspond to a set of segment features with interconnections that are densely distributed across the sequence. This type of feature set can encode interaction frequencies, which are a measure for the spatial distance between two segments. It is possible to measure the interaction frequencies of all possible segment pairs of a genome (Hi-C) or only interaction frequencies between one segment and all the other segments (4C). In terms of the previously defined feature types, it corresponds to a contiguous set of segment features of equal size that are interconnected with undirected interconnection features with a quantitative attribute. Exploring this kind of data can lead to valuable insights concerning the 3D structure of genomes under different conditions. Patterns of interest include compartmentalization for identifying active/inactive regions of the genome and topologically associating domains (TADs). We found visualizations of this type of feature in all kinds of layouts and parallel and orthogonal arrangements (see Table 4 ).

Linear Layout, Orthogonal Arrangement

From the beginning, orthogonal arrangements have been the standard in displaying genome-wide interaction frequencies. Each axis represents the same sequence and the spanned heatmap matrix encodes the interaction frequencies with color. By varying the bin size, heatmaps can represent the data and relevant features at different scales and different resolutions.

An interconnectivity visualization in an orthogonal arrangement can be understood as a genome browser-like tool with two axes instead of one to display interconnection features more effectively (see Figure 14 ). Similar to genome browsers, these tools can arrange multiple tracks in parallel, such as tracks showing gene annotations, epigenetic signals, or gene expression. As orthogonal layouts contain two axes, there can be one-axis foci, which correspond to the focus on each axis, and two-axis foci, which correspond to the foci in the matrix spanned by the axes. Table 4 shows both kinds of foci and scales, yet the main view configuration corresponds to the two-axes scales and two-axes foci.

Orthogonal arrangement in HiGlass [ KAL*18 ]. Interactions are encoded using a heatmap encoding. Additional tracks are aligned on both sequence axes.

Tools such as Juicebox and HiCExplorer are single view, single focus, single scale visualizations [ DRS*16 , WBN*18 ] (see Table 4 ). While HiCExplorer is a plotting tool which offers multiple different visualizations, Juicebox is more interactive and allows unlimited zooming. The web application Juicebox.js [ RTD*18 ] allows the visualization of multiple matrices using linked panning and zoom (n-1–1 view configuration).

The tool HiGlass is very flexible in terms of creating different views and offers unlimited zooming [ KAL*18 ] (n-n-n view configuration). Multiple matrices can be placed next to each other and navigation can be linked to compare features at different conditions. Moreover, it is possible to create overview-detail visualizations, where users can move and change the size of a sliding window placed in the main matrix to view the contents of the window in a detailed view. Multiple sliding windows can be created to visualize multiple foci.

HiPiler is a tool building on HiGlass for the analysis of regions of interest (ROIs) in genome interaction matrices [ LBK*18 ]. It displays ROIs as snippets , which are small, detailed regions of the matrix. Similar snippets are overlaid with a visualization of the average snippet on top of the pile. The arrangement of views and tracks of HiPiler is unique in our visualization taxonomy. Every pile of snippets represents a separate view, whereas each pile contains overlaid tracks, with a special track showing the aggregation on top. The snippet views are arranged by a clustering algorithm or a scatterplot. Snippets and the main matrix visualization are connected via brushing and linking. In contrast to the other presented matrix visualization, HiPiler combines pattern extraction with visual analysis in order to assess the quality of the extracted patterns.

Linear Layout, Parallel Arrangement

In our taxonomy the tool GIVE [ CYW*18 ] represents a unique arrangement of axes. Two axes are displayed in parallel and the same set of tracks is stacked on each axis. Interconnections are displayed as bands between the two axes in the two-axes track. Each axis can be navigated independently, therefore the visualization can display a different focus and scale on the single axes (see Table 4 ). Since interconnections can only be visualized between the two one-axis foci, there is only one focus and scale in the two-axes track. A possible reason why this arrangement of axes is rarely implemented for the analysis of Hi-C data is that when viewing the arrangements on a global scale, many bands overlap and it is hard to identify regions of interest to investigate further.

Linear Layout, Single Axis

Interconnectivity can be visualized in a linear layout with a single axis and is implemented in some genome browsers such as the WashU Epigenome Browser [ ZMX*11 ] and the 3d Genome Browser [ WSZ*18 ]. As described in section 4.2.2 , symmetrical matrices can be cut in half and rotated in order to display them as a track. Usually not the entire matrix is shown, but only interactions close to the diagonal to reduce the size of the track. Additional to its matrix visualization HiGlass also provides this kind of track [ KAL*18 ]. While this is useful to analyze short-range interactions in a genome browser like setting, usually long-range interactions cannot be analyzed with this type of visualization.

The 3d Genome Browser additionally implements another way of displaying Hi-C data in a linear layout [ WSZ*18 ], which is similarly implemented by HUGIn [ MXR*17 ]. Instead of showing interaction frequencies between all segments, they only show interaction between one segment and all other segments which corresponds to a virtual 4C plot. This is especially useful to visualize interactions between a biological relevant region, such as a gene, and all the other segments. However, selecting the biological relevant region requires previous knowledge of the users and only a part of the feature set can be explored at a time. Both tools encode Hi-C data as a line chart, where the y-axis corresponds to the interaction frequency. Moreover, other tracks showing additional data can be added and arranged in parallel.

A circular layout for the visualization of Hi-C data is implemented by Rondo [ TAKL*16 ], which uses a chord diagram to display interactions. As there are interactions between all segments, Rondo clusters individual interactions into larger groups. Every chromosome is encoded with a different color (see Figure 15 ). Rondo corresponds to a one view, one scale, one focus configuration and does therefore not allow viewing multiple circular plots of different datasets next to each other. However, it can encode for the comparison of two datasets that use the same coordinate system (both of the same species) by encoding the arcs corresponding to the different datasets with color and introducing a color for arcs in which the datasets overlap. In contrast to visualizations with an orthogonal axis arrangement it is not possible to compare more than two conditions. Rondo offers navigation and selection interactions. When zooming in, interaction clusters are progressively separated to show the interactions in more detail.

Circular layout in Rondo [ TAKL*16 ]. Chromosomes are encoded using color, bands represent interactions.

Spatial Layout

A 3D representation of genome structure displays reconstructions of the 2D profiles obtained using interaction-based methods such as Hi-C. They are deceptively reminiscent of protein structure visualization, depicting a single conformation of nuclear DNA. Tools such as HiC-3DViewer and 3DGB let users pan and zoom around a reconstruction and observe the proximity of likely neighbors [ DWZG17 , BMB*15 ]. They allow for the overlay of a gene annotation track and the extraction of data associated with genomic locations. Despite their capacity for displaying seemingly faithful representations of DNA structure in the nucleus, 3D visualization tools are severely limited by their reliance on algorithmic reconstructions of that structure. This hides information about heterogeneity and ambiguity in the underlying data and presents one of potentially many solutions to the constraints provided by contact mapping experiments.

5.2. Feature-Scale Visualizations

Features of the same type at regions within or across sequences can be summarized in order to find and visualize patterns. Three types of features are commonly summarized: (i) contiguous numerical features for the visualization of epigenetic signals at regions of interest, (ii) point features across samples for the visualization of genetic variants, and (iii) subsequences of the same length within a sample for the visualization of motifs, which are short, reoccurring subsequences that have a biological significance, such as a transcription factor binding site.

Summarizing features can decrease the amount and length of tracks that are displayed. Instead of locating, comparing, and browsing features within or across multiple tracks, the regions of interest are found by computational methods and displayed in one visualization. All summary visualizations displayed in Table 5 apply a linear layout, presumably since they usually show short sequences (single genes or only few bases) and do not display interconnection features, which often require the use of circular layout or an orthogonal arrangement of axes.

5.2.1. Non-Aggregated Feature Summaries

A common task when analyzing regions of biological importance, such as binding sites or genes, is exploring and summarizing the epigenetic signals around all instances of a particular type of a genomic feature, such as transcription start sites, protein-binding sites, or across the coding sequence of genes.

Feature-by-position heatmaps are a commonly used visualization technique that supports this task. They are constructed by assigning sequence windows around feature instances to tracks and arranging the tracks in parallel. The windows can be stretched to normalize across sequences of unequal length, such as coding regions for genes. Sequence positions are relative to the genomic position of each feature instance. The strength of the quantitative attribute is mapped to a sequential or diverging color map. Tracks can be ordered by mean or median signal across the columns of positions and can be clustered as well. A line chart, called a gene body plot often summarizes the heatmap, by showing an average value for each column of the heatmap. The stretching of sequence coordinates represents an exception to the track stacking described in section 4.2.2 , since the tracks are not aligned by sequence coordinates but stretched to align starts and ends of segment features of unequal size.

As each heatmap can only represent one epigenomic mark, multiple juxtaposed heatmaps are needed if more than one mark of interest should be visualized. Visualization tools that can generate these types of plots are deepTools [ RRG*16 ], which is both available through a Galaxy-based user instance and as a stand-alone command line tool, as well as the EnrichedHeatmap [ GESI18 ] R package (n-1–1 view configuration, see Table 5 ).

5.2.2. Aggregated Feature Summaries

Summarizing point features.

A genetic variant is a base alteration that can be evident for a specific phenotype of cancer. The basic goals of variant visualizations are exploring variant patterns together with protein domains, comparing the variants at different regions of the gene and correlating them with different disease phenotypes.

A commonly used visualization are lollipop plots which aggregate variants of a gene for an entire cohort (see Figure 16 [ CGD*12 ]). A lollipop plot consists of a valued point track and a valued segment track that are overlaid. The valued point track shows the number of variants (y-axis) at each position (sequence axis) for the entire cohort. For each mutation a dot is drawn at the corresponding position and abundance and the dot is connected to sequence axis with a line. Often the dots are annotated with the resulting protein change of the variant. The valued segment track shows different annotated protein domains.

Lollipop plot created using the cBio Portal [ CGD*12 ]. A lollipop plot is an example for the aggregation of sets of point features. Each dot represents a mutation at a given position in the amino acid sequence of the translated gene sequence (x-Axis) and with a corresponding abundance (y-Axis). The colored boxes correspond to protein domains.

A stand-alone tool for visualizing variants is Variant View [ FNM13 ]. In principle, it is similar to lollipop plots, but it shows additional tracks, such as further protein annotations. Moreover, it does not show the frequency of the variants. Each variantis shown separately with a detailed annotation, such as the mutation type and the change in the amino acid. In addition to the main visualization, it contains a weakly linked utility view in form of a table that contains further information about the displayed variants.

The tool MAGI is a multi view, single scale, single focus visualization (n-1–1, see Table 5 ) for population data, which shows mutation data and copy number data on sequence coordinates, as well as numerous other views for the visualization of non-sequence related data, such as meta data like age and gender [ LGH*15 ]. For the visualization of mutations, MAGI implements a visualization similar to a lollipop plot. Mutations are encoded by symbols and stacked at the corresponding positions. For each mutation, the shape of the symbol and its color, as well as its placement above or below the axis, encodes for information about the mutation type.

Summarizing Sequences

In contrast to variant visualizations, motif visualizations do not show base alterations that are evident of a disease or a phenotype. They show the composition of bases at a reoccurring sequence element, such as transcription factor binding sites. Motifs can be found by analyzing sequence patterns within a genome or across species. Motif visualizations are used to explore and summarize the base composition of short reoccurring biological relevant subsequences, or to compare the base composition at different positions within a motif.

Sequence logos, which are the basis of most motif visualizations, were introduced in 1990 [ SS90 ]. In order to construct a motif, the authors of the paper created an ungapped multiple sequence alignment of all possible sequence motifs and a table containing each base and its frequency at each position. For the visualization of the results they stacked the characters at each position on top of each other with the height of each letter corresponding to its entropy. In terms of our visualization taxonomy, this corresponds to a visualization of a single valued point track with features with complex attributes: each point is associated with the distribution of bases.

One of the first web applications to visualize sequence logos was implemented by Crooks et al. called WebLogo [ CHCB04 ] (see Figure 17a ). The tool pLogo uses the same basic visual encoding [ OCQ*13 ], but it displays values of statistical significance for each base at each position. Moreover, in addition to the motif, it shows underrepresented bases in a parallel track. Two Sample Logo has been developed for the comparison of motifs [ VIR06 ]. It visualizes statistically significant differences between motifs in three tracks: one for each track showing overrepresented symbols and one track showing consensus symbols.

Representations of the same sequence motif for a small sample of human intron-exon splice boundaries by (a) a sequence logo created with WebLogo [ CHCB04 ] and (b) a sequence bundle created with Alvis [ STK*16 ].

Sequence bundles represent an alternative to sequence logos [ KNS*14 ]. According to the authors describing sequence bundles, sequence logo visualization suffers from several limitations. Most importantly, these visualizations do not show relationships between residues at different positions. Sequence bundles display the features in an unconventional way, by encoding different bases with different positions on the y-axis. For each motif sequence of the multiple sequence alignment, a line is drawn (see Figure 17b ). At conserved positions, the lines will bundle together, while at variable positions, the lines will be more scattered. The tool Alvis is a Java implementation for the creation of sequence bundles [ STK*16 ]. Although sequence bundles enable visualizing longer sequences and relationships of residues at different positions, sequence logos remain the standard in visualizing motifs. A possible reason for this could be that the visual encoding is easier to interpret and the visualization is less cluttered.

MEME is a tool for the discovery of motifs and motif visualization [ BWML06 ]. It shows the location of found motifs in multiple input sequences as segment tracks and additional detailed views showing sequence logos for each motif (see n-n-1 view configuration in Table 5 ). The logos are associated with strongly connected utility views showing meta data about the motifs. Each motif is encoded with a colored rectangle in the segment tracks. When hovering over a segment the detailed views of the motif is displayed at the position of the cursor.

6. Multiple Genomic Coordinate Systems

6.1. genome-scale visualizations.

Sequences are often compared in order to find differences between two species in a field of research called comparative genomics . The evolutionary forces that shape genomes work on the scale of individual nucleotides in the form of mutations, insertions, and deletions, as well as on the scale of chromosomes in the form of duplications, translocations, and inversions. Sequences can be compared on different scales from nucleotide level to whole genomes, most commonly in linear and circular layouts and parallel, orthogonal and serial arrangements. The goal is to visually encode the differences between the sequences of the different genomes.

Tools for comparisons are mostly categorized in single view (1-1-1) or multi view, single scale, single focus (n-1-1) view configurations (see Table 6 ), as they show comparisons of one set of species in a single view or multiple comparisons in separate views. The only exceptions we found are GBrowse_syn [ MVS10 ] and Persephone [ Per ] as well as MizBee, which is an overview-detail visualization (n-n-1) in a circular layout [ MMP09 ].

Linear Layouts

In order to display long alignments, the online server VISTA offers tools to display alignments using line charts [ MBS*00 ]. In general, an alignment is used to identify conserved genomic regions in order to estimate the evolutionary relationship of organisms or finding shared genes. In VISTA each track corresponds to a sequence. The y-axis represents the percent identity with the reference genome computed with a window-averaged identity score. Regions with a high identity with the reference genome (conserved regions) which are part of an exon are indicated by highlighting the area under the curve dark blue, regions which are part of a non-coding part are colored red, and untranslated regions (UTRs) are colored light-blue. VISTA can display alignments of various lengths by adapting parameters, such as the zoom level, the resolution, and the minimum length of an aligned sequence in order to be displayed. A disadvantage of VISTA and other long alignments is that they cannot display segments that are conserved but have changed position during genome evolution (synteny blocks).

To highlight syntenic regions, other types of visualizations display similarities with connections via lines and bands or color. Cinteny [ SM07 ] applies both approaches but on different scales. On a chromosome scale, Cinteny connects syntenic regions with lines. When the entire genome is compared, chromosomes are displayed in a separate segregated layout (see Figure 19 ). One genome acts as a reference, where each chromosome is colored differently. Regions in the chromosomes of the other genome are colored corresponding to their syntenic region in the reference. This visualization demonstrates two of the shortcomings of color encodings for synteny. First, each of the human chromosomes is encoded using a different color. This is problematic since especially in the green spectrum the colors are hard to distinguish. Second, the visualization shows to which chromosome a synteny block of the mouse genome belongs, but not the exact position in that chromosome. The tool Syntenty Explorer implements this technique and two other possibilities of synteny visualization (described in the following paragraphs) [ BGM*17 ]. To conquer the problem of identifying the exact positions of synteny blocks in both genomes, Synteny Explorer allows adding lines to connect syntenic regions for one chromosome or an entire genome. However, when lines are added for an entire genome, the visualization becomes cluttered and hard to interpret.

Comparison of the human and mouse genome using Cinteny [ SM07 ]. Chromosomes are segregated and arranged in a linear separate layout, color encodes for syntenic regions.

In addition, Synteny Explorer implements a unique way of visualizing synteny [ BGM*17 ] by using animation to reorder syntenic regions to match the reference. We could not find this technique in any other sequence comparison tool, presumably because it is impossible to follow the movement of all syntenic regions in a genome or chromosome at once. However, Synteny Explorer represents an educational tool and animation can illustrate the actual process of rearrangements and inversions.

A straight forward solution for visualizing and comparing multiple long genomes is to provide more screen space as done by the tool BactoGenie [ ARJ*15 ]. BactoGenie visualizes genomic neighborhoods of bacterial strains on large displays to address scalability issues. Coding sequences are displayed as arrows, while the length and position of the sequence is preserved. Similarity is encoded using color, where similar coding sequence receive the same color.

Circular Layouts

Synteny can be visualized using Circos plots, where the two sequences are arranged in a circular combined layout. Synteny Explorer provides a visualization in this layout where synteny is shown by drawing bands between syntenic regions [ BGM*17 ].

In MizBee [ MMP09 ], Meyer et al. explores synteny relationships with a visualization using linked views at the genome, chromosome, and block levels (see Figure 20 ). MizBee represents the only comparison tool we found that applies multiple scales and multiple layouts. For the visualization of synteny on the genome level, MizBee uses a circular parallel arrangement, where one set of chromosomes is displayed in the outer ring (source chromosomes), the other set in the inner ring (destination chromosomes). A chromosome in the outer ring can be selected and inserted into the inner ring. Bands connecting the inserted source chromosome with destination chromosomes are drawn to show syntenic regions. Additionally, synteny is encoded using color. Regions and bands in the source chromosomes are colored according to their destination chromosomes. Two additional views in linear layouts show the selected source chromosome in detail, as well as a comparison of the source chromosome and a destination chromosome in a linear separated layout encoding synteny using bands.

MizBee visualizes synteny on different scales, applying circular and linear layouts and encodings using bands and color [ MMP09 ].

Orthogonal Layouts

In dot based approaches, similarities of sequences are indicated by diagonal rows of dots. They are used by tools like Gepard [ KAR07 ], EDGAR [ BKS*16 ] and Syn-Map2 [ HBSD*17 ]. These tools distribute the data along two dimensions to show a larger quantity of relationships than possible using a single axis (see Figure 21 ). All tools in an orthogonal layout are in a single view configuration, as most of them only show a single comparison between two genomes. Edgar Synteny Matrix [ BKS*16 ] shows multiple comparisons. However, all of them are aligned by sequence coordinates, or single view.

Dotplot created using SynMap2 [ HBSD*17 ] showing a comparison of the human and the mouse genome. The sequences are aligned in an orthogonal layout. Diagonal lines of dots imply syntenic regions.

6.2. Feature-Scale Visualizations

Comparing shorter sequences diverging by mutations, insertions and deletions is readily accomplished using alignment based visualizations. Tools such as Jalview, AliView and MSAViewer provide convenient implementations of this concept [ WPM*09 , Lar14 , YWR*16 ]. This form of visualization provides a detailed view of every sequence and general patterns of similarity can be identified by applying a categorical color scale to the four nucleotides. Since computationally aligning sequences creates a shared coordinate system, most of these tools are grouped into linear layout, no axis arrangement (see Table 6 ).

Not only the sequence itself but also other features can be compared across genomes at conserved regions. With the 3d Genome browser Hi-C data and epigenetic signals for biological relevant regions (such as genes) can be compared across species [ WSZ*18 ]. Since the regions are conserved, tracks for the different species can simply be stacked on the same sequence axis.

7. Discussion

The proposed taxonomies for data types, visualizations, and tasks span a very wide range of applications for genomic data. The most important takeaway is that despite the multi-scale nature of genomic data, not many tools take advantage of multiple linked view configurations that would support efficient navigation and pattern discovery of the space. A limitation of our taxonomy is that we do not distinguish between constant number of views, scales, and foci greater than 1 and a flexible number of views, scales, and foci. Both are currently represented as n in our taxonomy and tool review. Additionally, a more detailed evaluation and assessment of visual encodings that are used in tracks could be helpful in understanding where further visualization research is warranted. The same is true for utility views that are commonly integrated into visualization tools for genomic data. Utility views can apply visualization techniques, such as node-link diagrams and reorderable matrices for genomic data that do not visualize data in sequence context. However, as discussed in the Introduction, we did not include such techniques in our taxonomy given the limited spaces in this survey.

Our work shows that there are a lot of tools, not because there are many different visualization needs, but because the quantity of tools is driven primarily by the need to access a wide range of incompatible data formats and sources, as well as the need to integrate into common analysis workflows. These are also typically defined by the data formats that they operate on.

Our proposed taxonomies can be used to guide the development of a unified approach for visualizing genomic data, such as a grammar driven approach. However, the concerns about dependency on particular input formats, connections to analysis tools, and other external dependencies, certainly raises the question whether much would be gained by defining a generic visualization approach for genomic data. For such an effort to be successful, it would likely need to support an enormous number of data formats and sources, adding engineering overhead to the visualization challenge.

8. Opportunities and Challenges

The rapid progress in the development of new experimental assays to generate genomic data frequently leads to new visualization challenges and opportunities. Many challenges are related to complex genomic data such as 3D genome interactions, temporality, or scale of genomic data, in particular when the number of data sets or feature sets grows into thousands or more.

In particular, 3D genome interaction data present several unsolved challenges. In a recent review, Goodstadt et al. [ GMR17 ] report on challenges for visualizing 3D data in genome browsers. These data are multiscale (from the few microns of the nucleus to the few nanometers of nucleosomes), and multistate (hetero, euchromatin and several other states). They can also be time-dependent and need to capture the order of events describing how the genome structure changes over time due to biological processes. These introduce uncertainty in 3D genome interaction data. Subsequently, the challenges in visualizing these data include abstractions or reductions in data dimensions, variations to show changing events, finding meaningful patterns in the data or having proper interactions. For finding patterns or classification, side-by-side comparisons are common for 2D data, although this increases the cognitive load of the viewer. For 3D data, this is even more challenging as these contain large point-sets that are spatially distributed and may change over time. In certain cases, animation can be useful, although this also provides little insight when there are a large number of changes happening concurrently.

In addition to temporally resolved 3D genome structure data, it is now also possible to measure not only pairwise but also n-way interactions of regions across the whole genome. This will require novel visualization techniques that enable analysts to navigate this highly complex space and select lower-dimensional slices for visualization. A related challenge is the visualization and integration of imaging data (ranging from standard lightfield microscopy to super-resolution microscopy) with genomics data from chromosome conformation capture assays such as Hi-C. The mapping between image-based and genome-based coordinates and corresponding navigation will be particularly challenging in this context.

The number of genomes and patients in many studies will increase rapidly to thousands and in some cases, millions. This will require novel visualization techniques, as well as infrastructure (APIs, data formats, etc.) that allow visualization tools to access this data in an efficient and secure fashion. In the era of precision medicine, electronic health records and various types of sensor data will also need to be integrated with genomic data and this will be particularly important when viewing data from patient cohorts. These data are noisy, highly heterogeneous, and often have highly variable temporal resolution. Furthermore, driven by new insights into disease mechanisms, discovery of novel drug targets and therapeutics is an area which will benefit from the integration of visualization approaches for molecular data (e.g. MutationAssessor [ RAS11 ]) with variant and other genomic data viewers, as well as tools that aid in the prioritization of drugs and compounds.

There are also several technological challenges, that if addressed, would allow visualization tools to have a more profound impact in the analysis process. For example, tight integration of algorithmic and visual approaches and linking a diverse set of such tools into a coherent interface that reduces the cognitive burden of the investigator and enables seamless data flow across these tools. In particular in this context, more research is needed on evaluating genome visualization tools to guide future efforts and to develop user-centric approaches for building new visualization tools. Crisan et al. [ CMMG18 ] showed that a human-centered design approach integrating quantitative and qualitative feedback from users is important in creating successful clinical genomics reports, and suggested that this approach can also be useful in building complex bioinformatics data visualization software.

9. Conclusion

As our survey demonstrates, the direct impact of the data visualization field on genomic data visualization techniques has been limited to date. There are three possible explanations for the small number of publications on genomic data visualization that originated in the visualization community. First, this might be due to the size and complexity of the data that often require complex client-server architectures, resulting in engineering work that is not rewarded by the visualization community. This issue will become even more pressing in the context of visualizations for patient genomes which require special considerations of privacy and secure data access and storage. Second, genomics is a complex and rapidly evolving field which presents a steep learning curve for researchers who are not actively working in this domain. Third, as we demonstrate, the number of techniques and tasks is much smaller than the large number of published tools implies. Therefore, genomics might not be seen as a fruitful domain for visualization research projects.

However, the fact that the majority of genomic visualization tools is developed in an ad hoc fashion and published without formal task analyses or evaluations, is a clear sign that more visualization research is needed. On one hand, the bioinformatics community needs to become more aware of appropriate design and evaluation approaches for visualization tool development. On the other hand, the visualization community needs to be incentivized and enabled to study genomics visualization problems. This could be achieved by developing an infrastructure to provide convenient access to genomic data and to enable effective integration of new visualization tools into existing analysis frameworks or tool ecosystems. This will also allow researchers in the bioinformatics community to shift focus away from reimplementing basic functionality over and over and instead focus on visualization problems, rather than data access problems. Finally, as the first survey to comprehensively assess the landscape of genomic visualization tools, techniques, and tasks from a visualization point of view, our work should be considered a starting point for future efforts that aim at organizing the biological data visualization literature.

Multiple sequence alignment displayed with area charts using VISTA [ MBS*00 ]. Colors indicate if the conserved region belongs to an exon (dark blue), an untranslated region (light blue) or to a non-coding region (red).

10. Acknowledgements

The authors thank Peter Kerpedjiev for his contributions to the initial list of manuscripts and tools that are discussed in this survey. Funding for this project was provided by the National Institutes of Health (R00 HG007583, U01 CA200059).

Computer Graphics Forum published by Eurographics - The European Association for Computer Graphics and John Wiley & Sons Ltd.

• [ADG11]. Albers D, Dewey C, Gleicher M: Sequence surveyor: leveraging overview for scalable genomic alignment visualization. IEEE Trans. Vis. Comput. Graph 17, 12 (December 2011), 2392–2401. [ DOI ] [ PubMed ] [ Google Scholar ]
• [AHP18]. Amiryousefi A, Hyvönen J, Poczai P: IRscope: An online program to visualize the junction sites of chloroplast genomes. Bioinformatics (April 2018). [ DOI ] [ PubMed ] [ Google Scholar ]
• [ALS*15]. An J, Lai J, Sajjanhar A, Batra J, Wang C, Nelson CC: J-Circos: an interactive circos plotter. Bioinformatics 31, 9 (May 2015), 1463–1465. [ DOI ] [ PubMed ] [ Google Scholar ]
• [And09]. Anders S: Visualization of genomic data with the hilbert curve. Bioinformatics 25, 10 (May 2009), 1231–1235. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [ARJ*15]. Aurisano J, Reda K, Johnson A, Marai EG, Leigh J: BactoGeNIE: a large-scale comparative genome visualization for big displays. BMC Bioinformatics 16 Suppl 11 (August 2015), S6. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [AVPS*12]. Abeel T, Van Parys T, Saeys Y, Galagan J, Van de Peer Y: GenomeView: a next-generation genome browser. Nucleic Acids Res. 40, 2 (January 2012), e12. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [BGM*17]. Bryan C, Guterman G, Ma K, Lewin H, Larkin D, Kim J, Ma J, Farré M: Synteny explorer: An interactive visualization application for teaching genome evolution. IEEE Trans. Vis. Comput. Graph 23, 1 (January 2017), 711–720. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [BKS*16]. Blom J, Kreis J, Spänig S, Juhre T, Bertelli C, Ernst C, Goesmann A: EDGAR 2.0: an enhanced software platform for comparative gene content analyses. Nucleic Acids Res. 44, W1 (July 2016), W22–8. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [BM13]. Brehmer M, Munzner T: A multi-level typology of abstract visualization tasks. IEEE Trans. Vis. Comput. Graph 19, 12 (December 2013), 2376–2385. [ DOI ] [ PubMed ] [ Google Scholar ]
• [BMB*15]. Butyaev A, Mavlyutov R, Blanchette M, Cudrémauroux P, Waldispühl J: A low-latency, big database system and browser for storage, querying and visualization of 3D genomic data. Nucleic Acids Res. 43, 16 (September 2015), e103. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [Bro]. Broad Institute and the Regents of the University of California: Sashimi plot. https://software.broadinstitute.org/software/igv/Sashimi . Accessed: 2019-2-1.
• [BWML06]. Bailey TL, Williams N, Misleh C, Li WW: MEME: discovering and analyzing DNA and protein sequence motifs. Nucleic Acids Res. 34, Web Server issue (July 2006), W369–73. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [BYD*16]. Buels R, Yao E, Diesh CM, Hayes RD, Munoztorres M, Helt G, Goodstein DM, Elsik CG, Lewis SE, Stein L, Holmes IH: JBrowse: a dynamic web platform for genome visualization and analysis. Genome Biol. 17 (April 2016), 66. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [CGD*12]. Cerami E, Gao J, Dogrusoz U, Gross BE, Sumer SO, Aksoy BA, Jacobsen A, Byrne CJ, Heuer ML, Larsson E, Antipin Y, Reva B, Goldberg AP, Sander C, Schultz N: The cbio cancer genomics portal: an open platform for exploring multidimensional cancer genomics data. Cancer Discov. 2, 5 (May 2012), 401–404. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [CHCB04]. Crooks GE, Hon G, Chandonia J-M, Brenner SE: WebLogo: a sequence logo generator. Genome Res. 14, 6 (June 2004), 1188–1190. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [CMMG18]. Crisan A, McKee G, Munzner T, Gardy JL: Evidence-based design and evaluation of a whole genome sequencing clinical report for the reference microbiology laboratory. PeerJ 6 (January 2018), e4218. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [CSGB14]. Chelaru F, Smith L, Goldstein N, Bravo HC: Epiviz: interactive visual analytics for functional genomics data. Nat. Methods 11, 9 (September 2014), 938–940. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [CTB*09]. Carver T, Thomson N, Bleasby A, Berriman M, Parkhill J: DNAPlotter: circular and linear interactive genome visualization. Bioinformatics 25, 1 (January 2009), 119–120. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [CTYN15]. Cheong W-H, Tan Y-C, Yap S-J, Ng K-P: ClicO FS: an interactive web-based service of circos. Bioinformatics 31, 22 (November 2015), 3685–3687. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [CWC*13]. Cancer Genome Atlas Research Network, Weinstein JN, Collisson EA, Mills GB, Shaw KRM, Ozenberger BA, Ellrott K, Shmulevich I, Sander C, Stuart JM: The cancer genome atlas Pan-Cancer analysis project. Nat. Genet 45, 10 (October 2013), 1113–1120. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [CYW*18]. Cao X, Yan Z, Wu Q, Zheng A, Zhong S: GIVE: portable genome browsers for personal websites. Genome Biol. 19, 1 (July 2018), 92. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [CZ13]. Cao X, Zhong S: Enabling interspecies epigenomic comparison with CEpBrowser. Bioinformatics 29, 9 (May 2013), 1223–1225. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [DBG*17]. Dekker J, Belmont AS, Guttman M, Leshyk VO, Lis JT, Lomvardas S, Mirny LA, O’Shea CC, Park PJ, Ren B, Politz JCR, Shendure J, Zhong S, 4D Nucleome Network: The 4D nucleome project. Nature 549, 7671 (September 2017), 219–226. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [DLS*15]. Dhillon BK, Laird MR, Shay JA, Winsor GL, Lo R, Nizam F, Pereira SK, Waglechner N, Mcarthur AG, Langille MGI, Brinkman FSL: IslandViewer 3: more flexible, interactive genomic island discovery, visualization and analysis. Nucleic Acids Res. 43, W1 (July 2015), W104–8. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [Don09]. Donlin MJ: Using the generic genome browser (GBrowse). Curr. Protoc. Bioinformatics Chapter 9 (December 2009), Unit 9.9. [ DOI ] [ PubMed ] [ Google Scholar ]
• [DPH11]. Down TA, Piipari M, Hubbard TJP: Dalliance: interactive genome viewing on the web. Bioinformatics 27, 6 (March 2011), 889–890. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [DRS*16]. Durand NC, Robinson JT, Shamim MS, Machol I, Mesirov JP, Lander ES, Aiden EL: Juicebox provides a visualization system for Hi-C contact maps with unlimited zoom. Cell Syst 3, 1 (July 2016), 99–101. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [DWZG17]. Djekidel MN, Wang M, Zhang MQ, Gao J: HiC-3DViewer: a new tool to visualize Hi-C data in 3D space. Quantitative Biology 5, 2 (June 2017), 183–190. [ Google Scholar ]
• [ENC12]. ENCODE Project Consortium: An integrated encyclopedia of DNA elements in the human genome. Nature 489, 7414 (September 2012), 57–74. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [EYB*06]. Engels R, Yu T, Burge C, Mesirov JP, DeCaprio D, Galagan JE: Combo: a whole genome comparative browser. Bioinformatics 22, 14 (July 2006), 1782–1783. [ DOI ] [ PubMed ] [ Google Scholar ]
• [FBP*05]. Fraga MF, Ballestar E, Paz MF, Ropero S, Setien F, Ballestar ML, Heine-suñer D, Cigudosa JC, Urioste M, Benitez J, Boix-Chornet M, Sanchez-Aguilera A, Ling C, Carlsson E, Poulsen P, Vaag A, Stephan Z, Spector TD, Wu Y-Z, Plass C, Esteller M: Epigenetic differences arise during the lifetime of monozygotic twins. Proc. Natl. Acad. Sci. U. S. A 102, 30 (July 2005), 10604–10609. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [FGW*95]. Fraser CM, Gocayne JD, White O, Adams MD, Clayton RA, Fleischmann RD, Bult CJ, Kerlavage AR, Sutton G, Kelley JM, Fritchman RD, Weidman JF, Small KV, Sandusky M, Fuhrmann J, Nguyen D, Utterback TR, Saudek DM, Phillips CA, Merrick JM, Tomb JF, Dougherty BA, Bott KF, Hu PC, Lucier TS, Peterson SN, Smith HO, Hutchison CA 3rd, Venter JC: The minimal gene complement of mycoplasma genitalium. Science 270, 5235 (October 1995), 397–403. [ DOI ] [ PubMed ] [ Google Scholar ]
• [FNL16]. Freese NH, Norris DC, Loraine AE: Integrated genome browser: visual analytics platform for genomics. Bioinformatics 32, 14 (July 2016), 2089–2095. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [FNM13]. Ferstay JA, Nielsen CB, Munzner T: Variant view: visualizing sequence variants in their gene context. IEEE Trans. Vis. Comput. Graph 19, 12 (December 2013), 2546–2555. [ DOI ] [ PubMed ] [ Google Scholar ]
• [FPP*04]. Frazer KA, Pachter L, Poliakov A, Rubin EM, Dubchak I: VISTA: computational tools for comparative genomics. Nucleic Acids Res. 32, Web Server issue (July 2004), W273–9. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [FSB*12]. Fiume M, Smith EJM, Brook A, Strbenac D, Turner B, Mezlini AM, Robinson MD, Wodak SJ, Brudno M: Savant genome browser 2: visualization and analysis for population-scale genomics. Nucleic Acids Res. 40, Web Server issue (July 2012), W615–21. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [GCB*18]. Goldman M, Craft B, Brooks AN, Zhu J, Haussler D: The UCSC xena platform for cancer genomics data visualization and interpretation. May 2018.
• [GES16]. Gu Z, Eils R, Schlesner M: HilbertCurve: an R/Bioconductor package for high-resolution visualization of genomic data. Bioinformatics 32, 15 (August 2016), 2372–2374. [ DOI ] [ PubMed ] [ Google Scholar ]
• [GESI18]. Gu Z, Eils R, Schlesner M, Ishaque N: Enriched-Heatmap: an R/Bioconductor package for comprehensive visualization of genomic signal associations. BMC Genomics 19, 1 (April 2018), 234. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [GKA*11]. Gundersen S, Kalaš M, Abul O, Frigessi A, Hovig E, Sandve GK: Identifying elemental genomic track types and representing them uniformly. BMC Bioinformatics 12 (December 2011), 494. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [GMR17]. Goodstadt M, Marti-Renom MA: Challenges for visualizing three-dimensional data in genomic browsers. FEBS Lett. 591, 17 (September 2017), 2505–2519. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [Gol10]. Golub T: Counterpoint: Data first. Nature 464, 7289 (April 2010), 679. [ DOI ] [ PubMed ] [ Google Scholar ]
• [GS08]. Grant JR, Stothard P: The CGView server: a comparative genomics tool for circular genomes. Nucleic Acids Res. 36, Web Server issue (July 2008), W181–4. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [HACB*18]. Holm IA, Agrawal PB, Ceyhan-Birsoy O, Christensen KD, Fayer S, Frankel LA, Genetti CA, Krier JB, LaMay RC, Levy HL, McGuire AL, Parad RB, Park PJ, Pereira S, Rehm HL, Schwartz TS, Waisbren SE, Yu TW, BabySeq Project Team, Green RC, Beggs AH: The BabySeq project: implementing genomic sequencing in newborns. BMC Pediatr. 18, 1 (July 2018), 225. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [HBB*02]. Hubbard T, Barker D, Birney E, Cameron G, Chen Y, Clark L, Cox T, Cuff J, Curwen V, Down T, Durbin R, Eyras E, Gilbert J, Hammond M, Huminiecki L, Kasprzyk A, Lehvaslaiho H, Lijnzaad P, Melsopp C, Mongin E, Pettett R, Pocock M, Potter S, Rust A, Schmidt E, Searle S, Slater G, Smith J, Spooner W, Stabenau A, Stalker J, Stupka E, Ureta-Vidal A, Vastrik I, Clamp M: The ensembl genome database project. Nucleic Acids Res. 30, 1 (January 2002), 38–41. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [HBSD*17]. Haug-Baltzell A, Stephens SA, Davey S, Scheidegger CE, Lyons E: SynMap2 and SynMap3D: web-based whole-genome synteny browsers. Bioinformatics 33, 14 (July 2017), 2197–2198. [ DOI ] [ PubMed ] [ Google Scholar ]
• [HGW09]. Haque FN, Gottesman II, Wong AHC: Not really identical: epigenetic differences in monozygotic twins and implications for twin studies in psychiatry. Am. J. Med. Genet. C Semin. Med. Genet 151C, 2 (May 2009), 136–141. [ DOI ] [ PubMed ] [ Google Scholar ]
• [Hil35]. Hilbert D: Über die stetige abbildung einer linie auf ein flächenstück In Dritter Band: Analysis Âů Grundlagen der Mathematik Âů Physik Verschiedenes: Nebst Einer Lebensgeschichte, Hilbert D (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 1935, pp. 1–2. [ Google Scholar ]
• [HJ10]. Homann OR, Johnson AD: MochiView: versatile software for genome browsing and DNA motif analysis. BMC Biol. 8 (April 2010), 49. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [HJBN12]. Herbig A, Jäger G, Battke F, Nieselt K: GenomeRing: alignment visualization based on SuperGenome coordinates. Bioinformatics 28, 12 (June 2012), i7–15. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [HSD*14]. Hilker R, Stadermann KB, Doppmeier D, Kali-nowski J, Stoye J, Straube J, Winnebald J, Goesmann A: ReadXplorer—visualization and analysis of mapped sequences. Bioinformatics 30, 16 (August 2014), 2247–2254. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [KAL*18]. Kerpedjiev P, Abdennur N, Lekschas F, Mccallum C, Dinkla K, Strobelt H, Luber JM, Ouellette SB, Azhir A, Kumar N, Hwang J, Lee S, Alver BH, Pfister H, Mirny LA, Park PJ, Gehlenborg N: HiGlass: web-based visual exploration and analysis of genome interaction maps. Genome Biol. 19, 1 (August 2018), 125. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [KAR07]. Krumsiek J, Arnold R, Rattei T: Gepard: a rapid and sensitive tool for creating dotplots on genome scale. Bioinformatics 23, 8 (April 2007), 1026–1028. [ DOI ] [ PubMed ] [ Google Scholar ]
• [KAS*11]. Kharchenko PV, Alekseyenko AA, Schwartz YB, Minoda A, Riddle NC, Ernst J, Sabo PJ, Larschan E, Gorchakov AA, Gu T, Linder-Basso D, Plachetka A, Shanower G, Tolstorukov MY, Luquette LJ, Xi R, Jung YL, Park RW, Bishop EP, Canfield TK, Sandstrom R, Thurman RE, Macalpine DM, Stamatoyannopoulos JA, Kellis M, Elgin SCR, Kuroda MI, Pirrotta V, Karpen GH, Park PJ: Comprehensive analysis of the chromatin landscape in drosophila melanogaster. Nature 471, 7339 (March 2011), 480–485. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [KNS*14]. Kultys M, Nicholas L, Schwarz R, Goldman N, King J: Sequence bundles: a novel method for visualising, discovering and exploring sequence motifs. BMC Proc. 8, Suppl 2 Proceedings of the 3rd Annual Symposium on Biologica (August 2014), S8. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [KSB*09]. Krzywinski M, Schein J, Birol I, Connors J, Gascoyne R, Horsman D, Jones SJ, Marra MA: Circos: an information aesthetic for comparative genomics. Genome Res. 19, 9 (September 2009), 1639–1645. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [KSF*02]. Kent WJ, Sugnet CW, Furey TS, Roskin KM, Pringle TH, Zahler AM, Haussler D: The human genome browser at UCSC. Genome Res. 12, 6 (June 2002), 996–1006. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [KvEB*04]. Kerkhoven R, van Enckevort FHJ, Boekhorst J, Molenaar D, Siezen RJ: Visualization for genomics: the microbial genome viewer. Bioinformatics 20, 11 (July 2004), 1812–1814. [ DOI ] [ PubMed ] [ Google Scholar ]
• [KWS*15]. Katz Y, Wang ET, Silterra J, Schwartz S, Wong B, Thorvaldsdóttir H, Robinson JT, Mesirov JP, Airoldi EM, Burge CB: Quantitative visualization of alternative exon expression from RNA-seq data. Bioinformatics 31, 14 (July 2015), 2400–2402. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [KWZ*12]. Kong L, Wang J, Zhao S, Gu X, Luo J, Gao G: ABrowse–a customizable next-generation genome browser framework. BMC Bioinformatics 13 (January 2012), 2. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [Lar14]. Larsson A: AliView: a fast and lightweight alignment viewer and editor for large datasets. Bioinformatics 30, 22 (November 2014), 3276–3278. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [LB11]. Lajugie J, Bouhassira EE: GenPlay, a multipurpose genome analyzer and browser. Bioinformatics 27, 14 (July 2011), 1889–1893. [ DOI ] [ PubMed ] [ Google Scholar ]
• [LBK*18]. Lekschas F, Bach B, Kerpedjiev P, Gehlenborg N, Pfister H: HiPiler: Visual exploration of large genome interaction matrices with interactive small multiples. IEEE Trans. Vis. Comput. Graph 24, 1 (January 2018), 522–531. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [LGH*15]. Leiserson MDM, Gramazio CC, Hu J, Wu H-T, Laidlaw DH, Raphael BJ: MAGI: visualization and collaborative annotation of genomic aberrations. Nat. Methods 12, 6 (June 2015), 483–484. [ DOI ] [ PubMed ] [ Google Scholar ]
• [LJASH16]. Lerdrup M, Johansen JV, Agrawal-Singh S, Hansen K: An interactive environment for agile analysis and visualization of ChIP-sequencing data. Nat. Struct. Mol. Biol 23, 4 (April 2016), 349–357. [ DOI ] [ PubMed ] [ Google Scholar ]
• [LLL08]. Lemoine F, Labedan B, Lespinet O: SynteBase/SynteView: a tool to visualize gene order conservation in prokaryotic genomes. BMC Bioinformatics 9 (December 2008), 536. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [LSWQ17]. Lanchantin J, Singh R, Wang B, Qi Y: DEEP MOTIF DASHBOARD: VISUALIZING AND UNDERSTANDING GENOMIC SEQUENCES USING DEEP NEURAL NETWORKS. Pac. Symp. Biocomput 22 (2017), 254–265. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [LvBSD09]. Lajoie BR, van Berkum NL, Sanyal A, Dekker J: My5C: web tools for chromosome conformation capture studies. Nat. Methods 6, 10 (October 2009), 690–691. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [MBS*00]. Mayor C, Brudno M, Schwartz JR, Poliakov A, Rubin EM, Frazer KA, Pachter LS, Dubchak I: VISTA : visualizing global DNA sequence alignments of arbitrary length. Bioinformatics 16, 11 (November 2000), 1046–1047. [ DOI ] [ PubMed ] [ Google Scholar ]
• [MMP09]. Meyer M, Munzner T, Pfister H: MizBee: a multiscale synteny browser. IEEE Trans. Vis. Comput. Graph 15, 6 (2009), 897–904. [ DOI ] [ PubMed ] [ Google Scholar ]
• [MVS10]. McKay SJ, Vergara IA, Stajich JE: Using the generic synteny browser (GBrowse_syn). Curr. Protoc. Bioinformatics Chapter 9 (September 2010), Unit 9.12. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [MXR*17]. Martin JS, Xu Z, Reiner AP, Mohlke KL, Sullivan P, Ren B, Hu M, LI Y: HUGIn: Hi-C unifying genomic interrogator. Bioinformatics 33, 23 (December 2017), 3793–3795. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [NCB17]. NCBI Resource Coordinators: Database resources of the national center for biotechnology information. Nucleic Acids Res. 45, D1 (January 2017), D12–D17. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [NCD*10]. Nielsen CB, Cantor M, Dubchak I, Gordon D, Wang T: Visualizing genomes: techniques and challenges. Nat. Methods 7, 3 Suppl (March 2010), S5–S15. [ DOI ] [ PubMed ] [ Google Scholar ]
• [OBC*18]. O’Donoghue SI, Baldi BF, Clark SJ, Darling AE, Hogan JM, Kaur S, Maier-Hein L, McCarthy DJ, Moore WJ, Stenau E, Swedlow JR, Vuong J, Procter JB: Visualization of biomedical data. Annu. Rev. Biomed. Data Sci. 1, 1 (July 2018), 275–304. [ Google Scholar ]
• [OCQ*13]. O’Shea JP, Chou MF, Quader SA, Ryan JK, Church GM, Schwartz D: plogo: a probabilistic approach to visualizing sequence motifs. Nat. Methods 10 (October 2013), 1211. [ DOI ] [ PubMed ] [ Google Scholar ]
• [OKSF13]. Overmars L, Kerkhoven R, Siezen RJ, Francke C: MGcV: the microbial genomic context viewer for comparative genome analysis. BMC Genomics 14 (April 2013), 209. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [ORRL10]. O’Brien TM, Ritz AM, Raphael BJ, Laidlaw DH: Gremlin: an interactive visualization model for analyzing genomic rearrangements. IEEE Trans. Vis. Comput. Graph 16, 6 (November 2010), 918–926. [ DOI ] [ PubMed ] [ Google Scholar ]
• [OvHSF15]. Overmars L, van Hijum SAFT, Siezen RJ, Francke C: CiVi: circular genome visualization with unique features to analyze sequence elements. Bioinformatics 31, 17 (September 2015), 2867–2869. [ DOI ] [ PubMed ] [ Google Scholar ]
• [Per]. Persephone Software, LLC: Persephone: A multi-genome browser carefully crafted using latest technologies. https://persephonesoft.com/ . Accessed: 2019-2-2.
• [PMP*15]. Pavlopoulos GA, Malliarakis D, Papanikolaou N, Theodosiou T, Enright AJ, Iliopoulos I: Visualizing genome and systems biology: technologies, tools, implementation techniques and trends, past, present and future. Gigascience 4 (August 2015), 38. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [PSESVD10]. Petkau A, Stuart-Edwards M, Stothard P, Van Domselaar G: Interactive microbial genome visualization with GView. Bioinformatics 26, 24 (December 2010), 3125–3126. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [QLN*19]. Qu Z, Lau CW, Nguyen QV, Zhou Y, Catch-poole DR: Visual analytics of genomic and cancer data: A systematic review. Cancer Inform. 18 (March 2019), 1176935119835546. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [RAS11]. Reva B, Antipin Y, Sander C: Predicting the functional impact of protein mutations: application to cancer genomics. Nucleic Acids Res. 39, 17 (July 2011), 37–43. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [RCK*12]. Ryan MC, Cleland J, Kim R, Wong WC, Weinstein JN: SpliceSeq: a resource for analysis and visualization of RNA-Seq data on alternative splicing and its functional impacts. Bioinformatics 28, 18 (September 2012), 2385–2387. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [RRG*16]. Ramírez F, Ryan DP, Grüning B, Bhardwaj V, Kilpert F, Richter AS, Heyne S, Dündar F, Manke T: deeptools2: a next generation web server for deep-sequencing data analysis. Nucleic Acids Res. 44, W1 (July 2016), W160–5. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [RTD*18]. Robinson JT, Turner D, Durand NC, Thor-valdsdóttir H, Mesirov JP, Aiden EL: Juicebox.js provides a Cloud-Based visualization system for Hi-C data. Cell Syst 6, 2 (February 2018), 256–258.e1. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [RTRBH12]. Rogers MF, Thomas J, Reddy AS, Ben-Hur A: SpliceGrapher: detecting patterns of alternative splicing from RNA-Seq data in the context of gene models and EST data. Genome Biol. 13, 1 (January 2012), R4. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [SAB*16]. Strobelt H, Alsallakh B, Botros J, Peterson B, Borowsky M, Pfister H, Lex A: Vials: Visualizing alternative splicing of genes. IEEE Trans. Vis. Comput. Graph 22, 1 (January 2016), 399–408. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [SGPLB13]. Schroeder MP, Gonzalez-Perez A, Lopez-Bigas N: Visualizing multidimensional cancer genomics data. Genome Med. 5, 1 (January 2013), 9. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [Shn03]. Shneiderman B: The eyes have it: A task by data type taxonomy for information visualizations In The Craft of Information Visualization, Bederson BB, Shneiderman B, (Eds.). Morgan Kaufmann, San Francisco, January 2003, pp. 364–371. [ Google Scholar ]
• [SM07]. Sinha AU, Meller J: Cinteny: flexible analysis and visualization of synteny and genome rearrangements in multiple organisms. BMC Bioinformatics 8 (March 2007), 82. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [SS90]. Schneider TD, Stephens RM: Sequence logos: a new way to display consensus sequences. Nucleic Acids Res. 18, 20 (October 1990), 6097–6100. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [STK*16]. Schwarz RF, Tamuri AU, Kultys M, King J, Godwin J, Florescu AM, Schultz J, Goldman N: ALVIS: interactive non-aggregative visualization and explorative analysis of multiple sequence alignments. Nucleic Acids Res. 44, 8 (May 2016), e77. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [SW05]. Stothard P, Wishart DS: Circular genome visualization and exploration using CGView. Bioinformatics 21, 4 (February 2005), 537–539. [ DOI ] [ PubMed ] [ Google Scholar ]
• [TAKL*16]. Taberlay PC, Achinger-Kawecka J, Lun ATL, Buske FA, Sabir K, Gould CM, Zotenko E, Bert SA, Giles KA, Bauer DC, Smyth GK, Stirzaker C, O’Donoghue SI, Clark S: Three-dimensional disorganisation of the cancer genome occurs coincident with long range genetic and epigenetic alterations. Genome Res. (April 2016). [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [The10]. The International Cancer Genome Consortium: International network of cancer genome projects. Nature 464, 7291 (2010), 993–998. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [TRM13]. Thorvaldsdóttir H, Robinson JT, Mesirov JP: Integrative genomics viewer (IGV): high-performance genomics data visualization and exploration. Brief. Bioinform 14, 2 (March 2013), 178–192. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [VIR06]. Vacic V, Iakoucheva LM, Radivojac P: Two sample logo: a graphical representation of the differences between two sets of sequence alignments. Bioinformatics 22, 12 (June 2006), 1536–1537. [ DOI ] [ PubMed ] [ Google Scholar ]
• [WBN*18]. Wolff J, Bhardwaj V, Nothjunge S, Richard G, Renschler G, Gilsbach R, Manke T, Backofen R, Ramírez F, Grüning BA: Galaxy HiCExplorer: a web server for reproducible Hi-C data analysis, quality control and visualization. Nucleic Acids Res. 46, W1 (July 2018), W11–W16. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [Wei10]. Weinberg R: Point: Hypotheses first. Nature 464, 7289 (April 2010), 678. [ DOI ] [ PubMed ] [ Google Scholar ]
• [WKP*17]. Wang YE, Kutnetsov L, Partensky A, Farid J, Quackenbush J: WebMeV: A cloud platform for analyzing and visualizing cancer genomic data. Cancer Res. 77, 21 (November 2017), e11–e14. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [WNM14]. Wu E, Nance T, Montgomery SB: SplicePlot: a utility for visualizing splicing quantitative trait loci. Bioinformatics 30, 7 (April 2014), 1025–1026. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [WPM*09]. Waterhouse AM, Procter JB, Martin DMA, Clamp M, Barton GJ: Jalview version 2—a multiple sequence alignment editor and analysis workbench. Bioinformatics 25, 9 (May 2009), 1189–1191. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [WSZ*18]. Wang Y, Song F, Zhang B, Zhang L, Xu J, Kuang D, Li D, Choudhary MNK, Li Y, Hu M, Hardison R, Wang T, Yue F: The 3D genome browser: a web-based browser for visualizing 3D genome organization and long-range chromatin interactions. Genome Biol. 19, 1 (October 2018), 151. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [XRL04]. Xing Y, Resch A, Lee C: The multiassembly problem: reconstructing multiple transcript isoforms from EST fragment mixtures. Genome Res. 14, 3 (March 2004), 426–441. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [YCL12]. Yin T, Cook D, Lawrence M: ggbio: an R package for extending the grammar of graphics for genomic data. Genome Biol. 13, 8 (August 2012), R77. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [YN17]. Yardimci GG, Noble WS: Software tools for visualizing Hi-C data. Genome Biol. 18, 1 (2017), 26. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [YWR*16]. Yachdav G, Wilzbach S, Rauscher B, Sheridan R, Sillitoe I, Procter J, Lewis SE, Rost B, Goldberg T: MSAViewer: interactive JavaScript visualization of multiple sequence alignments. Bioinformatics 32, 22 (November 2016), 3501–3503. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• [ZMX*11]. Zhou X, Maricque B, Xie M, Li D, Sundaram V, Martin EA, Koebbe BC, Nielsen C, Hirst M, Farnham P, Kuhn RM, Zhu J, Smirnov I, Kent WJ, Haussler D, Madden PAF, Costello JF, Wang T: The human epigenome browser at washington university. Nat. Methods 8, 12 (November 2011), 989–990. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• View on publisher site
• PDF (2.8 MB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 17 February 2023

Hypothesis-free phenotype prediction within a genetics-first framework

• Chang Lu   ORCID: orcid.org/0000-0002-3272-0120 1 ,
• Jan Zaucha 2 ,
• Rihab Gam 1 ,
• Hai Fang 2 , 3 ,
• Ben Smithers 2 ,
• Matt E. Oates 2 ,
• Miguel Bernabe-Rubio 4 ,
• James Williams   ORCID: orcid.org/0000-0003-3206-7851 4 ,
• Natalie Zelenka   ORCID: orcid.org/0000-0002-1007-0286 2 ,
• Arun Prasad Pandurangan   ORCID: orcid.org/0000-0001-7168-7143 1 ,
• Himani Tandon 1 ,
• Hashem Shihab 2 ,
• Raju Kalaivani 1 ,
• Minkyung Sung 1 ,
• Adam J. Sardar   ORCID: orcid.org/0000-0002-6912-6125 2 ,
• Bastian Greshake Tzovoras   ORCID: orcid.org/0000-0002-9925-9623 5 ,
• Davide Danovi   ORCID: orcid.org/0000-0003-4119-5337 4 &
• Julian Gough   ORCID: orcid.org/0000-0002-1965-4982 1 , 2  

Nature Communications volume  14 , Article number:  919 ( 2023 ) Cite this article

7699 Accesses

2 Citations

6 Altmetric

Metrics details

• Computational biology and bioinformatics

Cohort-wide sequencing studies have revealed that the largest category of variants is those deemed ‘rare’, even for the subset located in coding regions (99% of known coding variants are seen in less than 1% of the population. Associative methods give some understanding how rare genetic variants influence disease and organism-level phenotypes. But here we show that additional discoveries can be made through a knowledge-based approach using protein domains and ontologies (function and phenotype) that considers all coding variants regardless of allele frequency. We describe an ab initio, genetics-first method making molecular knowledge-based interpretations for exome-wide non-synonymous variants for phenotypes at the organism and cellular level. By using this reverse approach, we identify plausible genetic causes for developmental disorders that have eluded other established methods and present molecular hypotheses for the causal genetics of 40 phenotypes generated from a direct-to-consumer genotype cohort. This system offers a chance to extract further discovery from genetic data after standard tools have been applied.

Similar content being viewed by others

The mutational constraint spectrum quantified from variation in 141,456 humans

Evidence for 28 genetic disorders discovered by combining healthcare and research data

A deep catalogue of protein-coding variation in 983,578 individuals

Introduction.

Sequencing of human genomes holds great promise for using genetic information to guide medical discovery and therapy. And yet in general, advances in our ability to extract useful information from genetic data are not being made as rapidly as advances in our ability to generate the data, leading to a growing imbalance of effort. Systematically predicting potential organism-level phenotypes or disease risks based on the information of a person’s genetic variation remains an unsolved challenge. The influence of common variants on phenotypes can be quantified by statistical weights from genome-wide association studies (GWAS) and presented as polygenic risk scores (PRS) 1 , 2 , 3 , while effects of rare variants can be expressed in terms of intolerance to high-penetrance functional variants in the human population 4 , 5 , 6 from both burden testing on common phenotypes and rare disease work in families. Nevertheless, these two leave a large area in the effect size – allele frequency space under-explored (Fig.  1a) , more explicitly not accounting for the influence of a significant amount of non-synonymous uncommon variants of medium or low penetrance 2 . On one hand, rare or low-frequency functional variants are under-interpreted in GWAS due to the inherent difficulties in the statistical evaluation of rare events in population genetics 7 , 8 . On the other hand, the statistical method for making gene-to-disease inference requires the support of multiple instances of high confidence predicted loss-of-function (pLoF) variants in a gene 4 , 9 , and it has been shown that such instances are rarely observed. It is estimated that cohorts roughly 1000 times bigger than gnomAD (which contains 125,748 exomes) are needed to gather evidence of their existence in most genes 10 . On a few datasets with extensive broad phenotyping, Phenome Wide Association Studies (PheWAS) 11 can leverage gene-based collapsing to address some rare variants. To further overcome these difficulties, and to apply to most datasets, we see the utility of a non-association-based method; ideally one incorporating the effects of rare or low-frequency variants, expanding our capability for discovery within the constraints of the existing scale of human genetics data.

a Positioning relative to heritability interpretation from two prevailing genetic association analyses 2 of many. G2P: Gene-to-phenotype databases, GWA: Genome wide association, PRS: Polygenic risk scores. The colour bar shows the genetic unit of analysis employed by each method. b Framework overview. The method takes an individual’s genetic data as input and produces a list of ontology terms for which the person is a potential outlier. It uses a large background of genomes to which the individual is compared, ontology databases with gene-phenotype relationships, and evolutionary intolerance of mutations in protein domain families encoded by hidden Markov models (HMMs). c Schematic illustration of the genetic landscape for an ontology term (HP:0000834 ‘abnormality of the adrenal glands’) highlighting genomes with high outlier scores. Each node represents a genome, and edges are proportional to genetic distance in eigenspace – in essence, a reduced dimensional feature space between genomes.

Existing non-associative approaches that investigate genotype-to-phenotype relationships mainly use supervised network models, usually learned from large genomic variant databases focusing on specific phenotypes, e.g. antimicrobial resistance 12 , 13 , yeast cellular phenotypes 14 , 15 and plant phenotypes 16 . These methods have demonstrated the possibility of using a knowledge-based strategy to make phenotypic prediction. However to achieve this, large datasets of millions of genetic variants (often through synthetic genetic arrays 17 ) are needed and such datasets are only available for selected phenotypes, and such supervised models are not applicable to complex human phenotypes. We propose an unsupervised knowledge-based system (Nomaly), that makes ab initio predictions of potential phenotypes from thousands of ontology terms (Fig.  1b ), leveraging the knowledge of protein domains through hidden Markov models 18 , 19 , 20 . Instead of being used to train a model at an early step, the phenotypes are used as a final step to evaluate which predictions performed significantly better than expected by chance. The underlying protein knowledge on which the ab initio models are based can be examined to provide molecular insights into the predicted phenotype, in other words unlike supervised models, the interpretability of our predictions is high.

The Nomaly system (Fig.  2 ) is built on the premise that a genetic extreme outlier can be defined that is predictive of an outlier in phenotype. Under this hypothesis, the system evaluates the genetic heterogeneity in the context of each phenotype. Consequently, not only is it able to consider the additive effect of multiple variants but also the non-additive combinatorial effect where some variants become relatively rare and deleterious in the presence of other more common variants. The challenging computation of this is made tractable via a linear algebra approach solving an eigenproblem (spectral clustering), described as segmentation-based object categorisation when used in image analysis 21 . A typical run includes a person or persons of interest and a large cohort-scale background, whereby outlier scores for thousands of terms in an ontology are calculated for each person of interest (Fig.  1b ). The outlier scores represent the likelihood of being an extreme outlier in the ontology-specific genetic landscape with respect to the chosen background (Fig.  1c ).

– see methods for detail. Genome data is inputted at the top and causal hypotheses are outputted at the bottom. In the orange top box (algorithm), firstly the functional distance between each missense variant is derived from domain-based HMM probabilities, scaled depending on zygosity (top row). Subsequently (second row) variants falling in the region of a gene with homology to an HMM representing a functional unit (domain), are collated into a genetic profile for a phenotype using domain-phenotype mappings inferred using dcGO 19 . This multi-domain collapsing of an ontology term can be likened to gene-based collapsing used in PheWAS 11 . Next (bottom row of the orange box) the profile of combined functional distances (from the top row) is used to calculate a genetic distance to every genome in the background. Spectral clustering of the distance matrix identifies which genomes are outliers under the profile (HP:0000834 in this illustration); nodes represent genomes and are coloured by outlier score (bottom right of the orange box). In the next (blue) box, only the top-scoring outlier phenotypes are passed to the confirmation stage, where some of these genetics-first predictions are identified as correct, giving a likely cause of the verified phenotype that was predicted.

To evaluate performance, here we wish to systematically assess: (i) whether there is a global consistency in statistics between the actual phenotypes and predictions based on outlier scores; (ii) whether the predictive success rate can be quantified; and (iii) how novel the predictions are. In this work, we describe a knowledge-based framework and demonstrate its significance and usefulness by answering these questions using three independent datasets, namely: a cohort of 2248 participants specially recruited for this study (DTC, below), the well-established dataset of 1133 children in the Deciphering Development Disorders (DDD) study with the respective gene-to-phenotype database DDG2P that provided genetic diagnoses for 40% of these children (majority through de novo mutations) 6 , 22 , 23 , and the Human Induced Pluripotent Stem Cells Initiative (HipSci) stem-cell bank 24 where there is the possibility to experimentally verify predictions on cellular phenotype.

Evaluation of the predictive power with a direct-to-consumer genetics cohort (DTC)

For the purpose of evaluation, we recruited a cohort of volunteers who had previously subscribed to direct-to-consumer (DTC) genotyping services (e.g. 23 and Me, AncestryDNA and others), or who were otherwise already in possession of personal genomic data files to participate in this study (Supplementary Fig.  1 ). To test the overall significance of outlier scores we presented each participant with a questionnaire asking them to self-identify, from a set of questions, any phenotypes from among 25 of their top-scoring ontology terms mixed equally with a further 25 top-scoring terms from a decoy – a randomly selected individual from the background (Fig.  3a ). The DTC cohort generated 2248 questionnaires, yielding 94,966 yes/no answers across 3672 ontology terms and 2086 written comments; see methods for details of QC. Questions were intended to identify only outliers, so if the question design resulted in a high positive response rate (>5%), they were excluded for identifying a common phenotype instead of an outlier. By requiring participants to self-identify, the often-costly challenge of phenotype data collection is simplified, however, we sacrifice accuracy compared to expert assessment, introducing noise that will mask the true predictive power by an unknown amount.

a Participants upload their DTC genome data on which outlier phenotypes are predicted, then shuffled with outliers from a decoy genome randomly selected from the background, to create a uniquely personalised questionnaire. Answers are used to confirm true predictions against decoys. b A test of 100,000 random permutations of the dataset shows that observed scores are on average higher for confirmed phenotypes, with a p -value of 8.25e-8 against randomly permuted scores. c The rate of identifying confirmed phenotypes by score threshold (blue) and number of above threshold predictions (green); at the default threshold of 0.022 the rate is more than double the rate for decoys, with a p -value of 7.93e-7 against 100,000 permutations. d The significance (green) of the top phenotypes by within-phenotype permutation of answers 100,000 times, and (blue) for the top x phenotypes, the number left after subtracting from the total those expected by chance. z -scores were derived from testing the null hypothesis that similar results can be obtained if scores are assigned randomly (see methods). p -value is calculated from z -score in a right-tailed hypothesis test. e For DDD patients, the 60 above-threshold predictions confirmed by clinical annotation with a p -value of 5.12e-4 versus data from 100,000 random permutations, using the same hypothesis test procedure as in d . Inset : the 50 patients with top predictions compared to published data 22 for whether a genetic diagnosis has been identified through DDG2P, and split by presence of de-novo mutation (DNM).

A statistical permutation test of outlier scores versus positive self-identification by participants proves that the method is significantly predictive of phenotype at the <1% level ( p -value: 8.25e-8, Fig.  3b ). The average rate at which participants self-identify a random phenotype was measured as 0.8% by taking answers to decoy questions. However, participants identify at a higher rate for predicted phenotypes and this increases monotonically with score threshold choice (Fig.  3c ). For high-scoring phenotypes the rate of self-identification is 1.8% ( p -value: 7.93e-7), but with improved phenotype measurement this could be increased. These results show that the high-scoring phenotype predictions harbour statistically significant signals and that about one half (ratio of above-threshold positive answer rate to decoy rate) of the top-scoring predictions, verified through self-identification, are due to underlying genetic variation identified by the algorithm.

If a confirmed prediction has a genuine genetic basis and has not occurred by chance, then other predictions for the same phenotype in other participants are more likely to be true. This non-independence can be exploited and measured by per phenotype permutation tests. Further permuting across all phenotypes corrects for multiple hypotheses. Of the 40 phenotypes with the top statistic by permutation (Table  1 and Supplementary Data  1 ), only 13.5 are expected to have occurred by chance (Fig.  3d ), which is a statistically much stronger result ( p -value: <10e-16) than when considering predictions independently as above. Examples of a novel gene, recovery of a known variant, a novel variant in a gene related to a known gene, and mechanistic explanations are shown below.

An assessment of potentially confounding factors (sex, ancestry and array type) establishes that the top phenotype predictions cannot be accounted for in this way. Although sex and three of the ancestry principal components (African, Gujerati Indians and Finnish) are predictive on the dataset, they are only significant ( p -value < 0.05 at Z -score > 1.65) on 4 of the top 40 phenotypes. More importantly, none of the variables correlate ( r  > 0.1) with outlier score on any of the top phenotypes. As a baseline comparison to outlier scores, association statistics calculated on the dataset (Supplementary Fig.  1 ) did not reveal any significant variants due to the small cohort size of each phenotype. Aggregated association scores at high FDR subjected to a permutation test (Fig.  3d ) identify 6(±2) out of a top 21 phenotypes having a significant variant association (Supplementary Data  5 ) not expected by chance at Z -score 2.9. The removal of points from the data for questions with outlier score >0.022 results in total loss of significance, implying that genetics-first selection of questions improved the power of the association results.

Potential genetic diagnoses for Deciphering Developmental Disorders cohort children (DDD)

In addition to the evaluation on our own DTC cohort for significance, a comparison was made to state-of-the-art work in the field on the well-established DDD cohort for the purpose of assessing novelty. DDD consists of 1133 trios with developmental disorders who have been exome-sequenced and annotated with Human Phenotype Ontology (HPO) 25 terms by clinicians 6 , 22 , 23 .

Predictions on DDD were restricted only to HPO terms relevant to developmental disorders that were used for annotation by clinicians. Of these, 60 predictions above threshold matching clinical annotations for 50 patients were found (Fig.  3e ). This rate is slightly lower, but consistent with the DTC results. Comparison to the published list of clinical diagnoses 6 , 22 shows that 62% (31) of these patients had received no genetic diagnosis, including 15 who have no de novo mutations (DNM); the published diagnoses in the DDD paper were made through DNM missense variants, inherited variants and rare chromosomal events in known genes using the developmental disorder gene-to-phenotype (DDG2P) database. Thus, plausible genetic explanations can be discovered for families not covered by the established G2P interpretation method (Supplementary Data  2 ).

A global analysis of the predictions made on DDD that match clinical annotations confirms significance of the method ( p -value: 3.62e-4), although it is less than for the DTC cohort due to limited data, restricted to developmental disorder-specific HPO terms. Likewise, about 1/3 of the above-threshold predictions ( p -value: 5.12e-4) are expected to be true instead of about 1/2 in DTC (subtracting decoy rate from prediction rate). Examples of a novel variant in a known gene and of a combinatorial effect are shown below. We conclude that, not only are the predictions significant on an independent dataset, but also largely non-redundant to those made by existing state-of-the-art methods, thus advancing the field.

Application to interpreting genetic variants for cellular phenotypes on a large panel induced-pluripotent stem cell lines (HipSci)

Although HPO is the most common ontology used to annotate human phenotypes, and that used by DDD 22 , the DTC cohort study also included several other mammalian and disease ontologies (Supplementary Fig.  1b ) and the gene ontology (GO) 26 . Despite being the ontology richest in data, GO performed worse on DTC than the other ontologies ( p -value: 2.15e-2 for GO and p -value: 6.05e-7 for non-GO using threshold). This is presumably due to the difficulty in self-identifying molecular and cellular level terms, especially without recourse to invasive measurements on the person. The HipSci project provides exome sequence data for hundreds of iPS cell lines from different individuals, and thus offers an opportunity to examine this prediciton framework in the application to molecular and cellular phenotypes instead of patient-level phenotypes.

Outlier scores were generated for GO terms from the exome sequences corresponding to 427 HipSci samples, generating predictions potentially relevant to cellular phenotype. It is not possible to use this dataset to assess global performance versus phenotype identification as with the other two cohorts, but the top-scoring phenotype terms were explored to identify a prediction that could be empirically validated in vitro. The phenotype “negative regulation of centrosome duplication” within the “biological process” domain of GO presented high-scoring predictions (see below) for some cell lines, and was selected principally for its suitability for measurement in iPS cells with an existing assay.

Centriole staining was carried out on HipSci cell lines corresponding to all five individuals predicted to be outliers from their exome sequence, and three controls not predicted to be outliers. Counting of the centrioles indicated that three out of five predicted cell lines displayed an elevated percentage of cells with more than two centrioles, suggesting defects in centriole regulation and cell cycle, confirming the accuracy of the prediction (Fig.  4 ). This phenotype would not be identifiable from symptoms in the DTC or DDD cohorts, but this empirical evidence demonstrates the value of the approach for discovery at the cellular or tissue level.

This refers to 'any process that decreases the frequency, rate or extent of centrosome duplication'. a Representative examples of each of the indicated cell lines. Centrioles were detected by staining with gamma-tublin (γ-tub), nuclei were stained with DAPI. Asterisks indicate cells with more than two centrioles. The Hoik-1, Sehp-2 and Kegd-2 were control cell lines. b Histogram showing the percentage of cells with more than 2 centrioles per cell in the indicated cell lines. Results are summarised as the mean ± s.e.m. from 3 independent experiments (600-800 cells per cell line were analysed; each percentage from each experiment were shown as dots; *: P  < 0.05; **: P  < 0.005; ns: not significant). Specifically, the p -values are: Boqx-2 P  = 0.045, Suul-1 P  = 0.0021, Yoch-6 P  = 0.0023 (one-sided t-test).

Types of genetic outlier that explain potential phenotype outlier

An outlier in the genetic landscape of an ontology term, as detected by this approach, can be caused by a single or by multiple rare variants. In the case of multiple variants, each variant can be classified according to whether it is absolutely required to achieve a score above the threshold in the cohort, or whether it merely contributes to an above-threshold score; alternative variants can achieve above-threshold scores in different people (Fig.  5a, c ). There can also be a combinatorial effect contributing to the score (e.g. Fig.  6g ), whereby a variant becomes relatively rarer and more deleterious in the presence of a particular genotype consisting typically of a few common variants, exemplified here by having a higher outlier score if classified into a cluster (Fig.  1c ) than if there is no cluster. The most common case (71%; Figs.  5b , 1-a/b) is where a single variant that is highly deleterious is sufficient to achieve the threshold although others may contribute additional score (44% cases; 1-b). Multiple variants being required to cause an outlier account for 29% of cases (2-a, 2-b). Combinatorial effects are important in a small minority of cases (Fig.  5d ).

a Outliers classified by underlying genetic variants into 4 types: 1-a, single variant only required; 1-b, single variant plus contributing variant(s); 2-a, multiple variants but dominated by one high-scoring variant; or 2-b, multiple variants required. b Distribution of the four types of outliers in the DDD cohort. c Violin plots to show the distribution of log-scaled minor allele frequencies (MAF) of variants involved in all predicted outliers, and by type of outliers in the DDD cohort. Colours show different types of outliers, schemes show roles of the variants involved, similar in a . Specifically, all (grey): 3682 variants involved in at least one outlier prediction, 539 variants involved in type 1-a outlier (dark blue), 601 required (star) and 1548 contributing (triangle) in type 1-b (light blue), 416 required (star) and 20 contributing (triangle) in type 2-a (orange), and 1767 in type 2-b (red). Distributions were generated using a kernel density estimate in the seaborn package 47 . Boxes show quartiles and whiskers represent 1.5 multiple of interquartile range. d Percentage of outliers with a combinatorial component to the score with variants contributing non-independently.

Any coordinates shown are relative to genome assembly GRCh37. a Novel gene association . ADAM7 [ https://www.ncbi.nlm.nih.gov/gene/8756 ], ADAMTS13 [ https://www.ncbi.nlm.nih.gov/gene/11093 ]. b Known variant . Chr6 Pos26093141, HFE-C282Y. c Novel variant in related gene . Chr12 Pos52913668, KRT5-G138E. d Single variant . Chr17 Pos29586054, NF1-L1425R. e Single variant . Chr19 Pos17927755, INSL3-R102C. f Novel variant in known gene . Chr3 Pos18143037, SOX2-L75P. g Combinatorial effect . 2 variants in CYP4B1-R375C/R340C and CYP2A7-T347A and CYP2D6-R245C. h Experimentally validated on HipSci . Chr3 Pos48414274, FBXW12-P6L. Panels (a) and (f) were created with BioRender.com.

The majority (90%) of single-variant-based predictions were caused by a rare variant with minor allele frequency (MAF) <0.5% for a heterozygous genotype, or MAF < 1% for a homozygous genotype. However, not all rare variants result in a positive prediction, for example if it is not highly deleterious, or when many people from the chosen background harbour different rare variants for the phenotype, suggesting the ontology term is not highly evolutionary constrained. In multiple-variant-based positive predictions, 24% of variants are rare, and 55% are low-frequency (MAF < 1% heterozygous or MAF<5% homozygous) (Fig.  5c ). Whilst our approach does not filter variants based on allele frequency (as illustrated in Fig.  1a ), there is insufficient power to recover the effects of common variants given the size of the DDD or DTC cohorts. As each phenotype question is only presented to a small subset of participants, only effects of large magnitude emerge from rare genotypes.

In Fig.  6 we show some examples of different types of discovery from the results. (1) Novel gene association. The representation in Fig.  6a shows how primary evidence from mouse knock-out experiments in genes ADAM9/17/19 led to a dcGO 18 , 19 link between a mitral valve phenotype and the reprolysin-like domain shared by these genes. Novel variants identified in respective domain of human genes (ADAM7, ADAMTS13) confirmed the mitral valve phenotype via questionnaire. (2) Known variant. In Fig.  6b a variant used to predict confirmed hemochromatosis in DTC participants was already known in ClinVar 27 . The diagram shows left to right: how multiple genes (including HFE) labelled with the ontology term were used by dcGO 18 , 19 to link it to domain families, then variants in DTC participants within these domains prioritised by HMM probabilities led to prediction. Had the HFE variant not been known (HFE not included as known), it is possible with this approach that it would have been discovered de novo by the DTC study on the basis of domain association. (3) Novel variant in related gene. The prediction in Fig.  6c includes other variants in keratin genes KRT75, KRT78 not previously linked to this ontology term. This variant is found in the head region (1–167) of the intermediate filament rod domain adjacent to an ELM motif involved in a protein-protein interaction with an SH3 domain. This figure shows a structural interpretation of the mutation of G138 to a polar negatively charged Glu disrupting binding to the electrostatic surface of SH3 domain from PDB structure 2GBQ.

Single variant. The single rare and functional variant (Fig.  6d ) in the neurofibromin (NF1) gene is sufficient to produce the top-ranked score for this phenotype in DTC. The image shows a structural interpretation of the central domain of neurofibromin (with helices 6c and 7c in cyan and yellow respectively) bound to Ras (surface model, top) and the Leu residue at 1425 rendered as spheres. Substituting Arg for Leu at this position will disrupt the helical geometry and successful interaction with Ras. (4) Single variant. In Fig.  6e this single rare and functional variant in the insulin-like 3 (INSL3) gene is sufficient to produce the top-ranked score for this phenotype in DTC. The primary structure shows the variant lying within a conserved polar charged patch. The PDB structure 2H8B shows the two disulphide bonds that stabilise the structure. A mutation of the Arg to Cys could disrupt the polar charged region of the protein and even interfere with the correct formation of disulphide bonds. (5) Novel variant in known gene. Two independent cases in the literature of missense mutations in SOX2 (G130A and A191T) mean the example in Fig.  6f is already a known gene for the phenotype in the OMIM 28 database. The phenotype was correctly predicted by our work on a DDD patient due to a novel variant in the HMG-box domain of this known protein. (6) Combinatorial effect. The phenotype in Fig.  6g that was correctly predicted on a DDD patient requires 2 variants in CYP4B1 (in linkage disequilibrium), plus additional variants. There is a combinatorial component to the score which raises this patient to the top rank in DDD for this phenotype. This Venn diagram shows that common variants are observed to co-occur (shaded areas) with the CYP4B1 variants much less than expected if the variants were independent (intersection areas). The top-ranked patient has all four variants. (7) Experimentally validated on HipSci. Figure  6h shows a homology model for F-box/WD repeat-containing protein 12 (FBXW12) using PDB structure 1NEX as template shows the proline at residue 6 that mutates to leucine in the variant, residing within the N-Proline box motif in N-terminal F-box domain. A proline at the N-cap position mediates hydrophobic interactions similarly to other N-cap residues Asn, Asp, Ser, and Thr. The mutation to leucine in this position impacts interactions within the motif, likely causing a change of specificity and/or affinity.

This paper describes a framework for performing and evaluating hypothesis-free phenotype prediction directly from a human genome. The key value is in providing potential genetic explanations for phenotypes that have been confirmed in the individual, which due to the high novelty and link to mechanism of the output, has potential for application to genetics-led drug target identification. Studying the combined effects on complex phenotype across variants in multiple genes is often impossible with simple statistical models, because of the lack of statistical power on existing cohort sizes. Also due to the lack of statistical power, rare or low-frequency variants are under-interpreted in GWAS 7 , 8 . An ab initio model can partially overcome this limitation by testing a statistically small number of causally deduced direct predictions (a genetics-first approach). This knowledge-based framework with in silico and experimental validation approach described here has been shown to achieve this for exome missense variants. Our method, therefore, offers the ability to evaluate the effect of rarer genetic variants in a combinatorial way through linear algebra where statistical associative methods would not be applicable.

Hypothesis-free phenotype prediction with this genetics-first approach could be applied in principle to other ab initio models, but we chose to deploy a model based on protein domains, which are the functional units of proteins. Hidden Markov models built on protein domains 18 enable the quantification of structural and functional effects of variants, and our domain-centric gene ontology (dcGO 19 ) resource provides the link to phenotype. The domain-based model emphasises potential for novelty over coverage of known genes by carrying over the functional property of the domain across many genes.

Established genetic cohorts do not lend themselves well to testing a genetics-first approach since data is mostly only available for a restricted list of hypothesis-derived phenotypes – and usually not encoded in ontology terms, although increasingly attempts are being made to widen phenotype capture, e.g. using ICD-10 29 . The DDD cohort is one of the early large-scale studies to adopt HPO terms and was included in this analysis as a known reference point in the field. However, to truly test the genetics-first approach, we need an interactive cohort of participants. Participants were recruited for their willingness to provide their genotype data first, then respond to personalised phenotype data collection post-prediction. Evaluation on both cohorts similarly confirms the predictions as highly significant yet also characterises the predictions as having a high false-positive rate; expectations are that for roughly 1/3–1/2 of confirmed high-scoring predictions, the causal genetic explanations will be true. Combining results per phenotype is more powerful, showing that explanations for about 26 of the top 40 confirmed phenotypes are likely to be true (Fig.  2d ). The predictions were also well-differentiated from results obtainable with other methods, confirmed by comparison to that achieved by DDD annotations of DNMs.

This method offers a powerful discovery tool for hypothesis generation from genetic data. The tool does not replace or compete with existing tools for human genetics which are largely aimed at being clinically actionable or offering effective intervention strategies. It rather complements and adds to them, aiming to enable medically high-value scientific discoveries first suggested from the analysis of genomic data. Independent validation of the hypothesis generated by the prediction with relevant assays is recommended. The expectation is that genes/variants will be mostly novel, and characterised by high magnitude of effect often from rare variants, sometimes severally, and occasionally acting combinatorially. In its essence, the tool is a genetic outlier detector, so it is important to consider the background with respect to which the individual is an outlier. In this work, we used the 1000 Genomes Project phase 3 (1KGP3) 30 , 31 genomes as a diverse background with representation of all major ancestry in the DTC cohort, but by selecting a different background, the definition of an outlier can be adjusted to suit the desired research question.

The catalogue of high-scoring results from the analysis of the three cohorts: DTC, DDD and HipSci provide over a hundred putative causal genetic explanations for numerous developmental, cellular and human/mouse/disease phenotypes. In addition to the supplementary data tables, they are also provided online with an interface to aid browsing and searching the variants, their classification of type, genes, scores and phenotypes ( https://supfam.org/nomaly ), also including the database of 5,857 ontology questions as a resource for others.

The framework could also be used by other functional effect predictors, model organism ontologies and including known genes (to recover more of the less novel variants). We showed that even simple association statistics can be used within the framework, but that question selection is important, suggesting that a genetics-first step could be used to increase the power of PheWAS studies. Having proven principle on missense variants, we expect the future growth in this area by us and others will be by extension to other mutations e.g. indels and non-coding variants.

To sum up, the traditional approach to human genetics, where we ask “Does the data contain the answer to my question?”, has been turned on its head, and we instead ask: “For which questions does an answer lie within the data?”.

Ethics declarations

The DTC cohort study was granted ethics approval by the University of Bristol, via the Faculty of Engineering Research Ethics Committee with approval ID number 539500 (project ID 361 and amendment 2322). Informed consent from all participants was obtained. HipSci Lines samples were collected from consented research volunteer recruited from the NIHR Cambridge BioResource through ( https://www.cambridgebioresource.org.uk ). Initially, 250 normal samples were collected under ethics for iPSC derivation (REC Ref: 09/H0304/77, V2 04/01/2013), which require managed data access for all genetically identifying data, including genotypes, sequence and microarray data (‘managed access samples’). In parallel the hipsci consortium obtained new ethics approval for a revised consent (REC Ref: 09/H0304/77, V3 15/03/2013), under which all data, except from the Y chromosome from males, can be made openly available (Y chromosome data can be used to de-identify men by surname matching), and samples since October 2013 have been collected with this revised consent (‘open access samples’). The majority of samples were European. Work performed in the laboratories has been compliant with the Institutional Review Board directives for the experimental work and use of data.

Nomaly framework

The framework consists of two primary parts: the predictive algorithm and the confirmation of predictions (Fig.  2 ). The algorithm takes an individual genetic data file (e.g. SNP array or exome) as input (top of Fig.  2 , input to orange box) and outputs phenotypes for which the individual is predicted to be an outlier (from orange box into blue box). The definition of ‘outlier’ is made relative to a background comprising thousands of genomes (bottom left of orange box). For the DTC and HipSci cohorts, the 1KGP3 genomes were used as a background, but for individuals in the DDD cohort, the cohort itself serves as the background. There are thousands of potential phenotypes, taken from 17 biomedical ontology databases, each assigned a score (below) for how much of an outlier it is for the individual in question against the background. In principle any genome or any biomedical ontology database can be used for a given study. Hidden Markov models (HMMs) are used to estimate deleteriousness; models from the domain databases SUPERFAMILY 18 and Pfam 32 were used, but in principle any HMMs can be used or any other measure of deleteriousness from a variant effect predictor.

The confirmation step (blue box in Fig.  2 ) is the assessment of whether an individual is indeed an outlier for a phenotype suggested by the genetics-first analysis. In principle the assessment can be carried out in any way that lends evidence to confirm the outlier (the DDD study used at least two certified clinical geneticists to perform assessment 22 ), but in this work automated matching of ontology terms was used. Predictions from the first step that are subsequently confirmed in this step become candidate hypotheses (output of blue box) linking a phenotype to variants via protein domains in genes.

Outlier scores (orange box in Fig.  2 ). The predictive algorithm includes (1) quantification of consequence of missense variants using evolutionary intolerance to the amino acid substitution in a protein domain, done by taking the difference in amino acid emission probability (analogous concept to FATHMM 33 ) and (2) generating variant lists for ontology terms through domain-centric linking to phenotypes (taken from dcGO) 18 , 19 . The mapping from variant to amino acids in proteins for (1) is done using the Variant Effect Predictor (VEP) tool 34 (N.B. VEP used only for genomic mapping and no other functionality or scores are used) and mapping to domains using HMM sequence matching. The mapping of domains to ontology terms in (2) is combined with the variants falling within them from (1) to give a list of variants, commonly across multiple proteins, for each ontology term. Each ontology term is processed independently.

For a given term, a total genetic distance may be calculated between two individuals by summing the individual distances (defined as the log odds ratio of HMM probabilities for the two amino acids from the Dirichlet mixture) for all variants in the list for which they differ, depending on zygosity; distances are increased fourfold when both are homozygous and opposite. The all-against-all distances between the members of the background and the individual, and between each other, can be used to construct a distance matrix. This matrix is then translated to a similarity matrix through a Gaussian kernel and used as the input to spectral clustering to determine whether there is hidden structure in the genetic landscape of that term, namely by identifying the biggest gap in eigenvalues 21 . If no hidden structure is found, then the individual’s outlier score will be equivalent to the average Euclidean distance to members of the background; in this case individuals with very rare and highly deleterious genotypes will have a high outlier score (Fig.  5c ). If a hidden structure is found by spectral clustering, K-means is then performed on the reduced-dimensional space derived from the top eigenvectors selected by the elbow method. In this scenario the outlier score becomes the sum of the local distance (from the cluster) and the global distance (between clusters) (Fig.  1c ). To normalise for cluster size the global distance is multiplied by μ where:

where γ specifies the penalty strength for large clusters; it was set to 9 in this study, conferring >99% penalty for large clusters with over 50% of the entire cohort.

Finally, since phenotypes have very different score distributions, a transformation (first described in Zaucha et al 35 ) is employed to generate a universal score function comparable between phenotype terms.

where s trans ( p ) is the transformed score of participant p in a given term, s ( p ) is the original score and s rank ( p ) is the rank of the original score within the Term. φ determines the strength of contribution of the rank to the total term score; it was set to 150 in this study, conferring the top ~2% a significant contribution.

Combinatorial component

The non-linear method of clustering used can result in one or more variants being markedly rarer within one cluster relative to the entire background after genomes with other shared genotypes are grouped together. Thus a genome with a high global score due to the variants common to the cluster, and a high local score due to a variant rare only to that cluster, will have a higher total score than it would from its average Euclidian distance from the background. This additional score from variant rarity only increases in the presence of other specific variants, and is defined as the combinatorial contribution (as in Fig.  5d ).

Computational cost

Calculating spectral clustering requires a large computation, even when using low-level linear algebra BLAS 36 libraries run on many threads in parallel. This is because it involves solving eigenproblems of a large matrix, which do not scale linearly with matrix size. For guidance, 5,800 terms on a batch of 50 SNP array genotype files with 2,504 background genomes (2554 x 2554 matrix) takes 21 hours with 12 threads (9.5 hours with 48 threads). Solving ~2,000 terms (independent eigenproblems) on a 3600x3600 matrix of exomes (including background) takes 3 hours using 600 threads.

Recruitment

Participants with access to personal direct-to-consumer genotype data were recruited anonymously online. Some participants were recruited in collaboration with OpenSNP 37 and (separately) Sano Genetics. In this cohort 2,248 participations were recorded, with the participant website accessed from all over the world (Supplementary Fig.  1a ). DTC genome data uploaded by participants was processed into a homogeneous format and quality-controlled with GenomePrep 38 , which also detects the sequencing method/genotyping array version. Imputation was not used for the analysis presented in this paper, although the main result of Fig.  3 was replicated on data imputed from the DTC genotype data (Supplementary Fig.  3 ) to confirm that there was little difference – only 4 terms outside the top 50 entered the top 40, with another 4 in the top 50 making a minor change moving up to the top 40. The IMPUTE2 39 package was used with help from VCFtools 40 and BCFtools by SAMtools 41 . A similarity matrix of all genomes in the cohort was calculated, consanguineous relationships were recorded and genetic duplicates removed; people submitting independent files from multiple providers were only allowed to participate once (Supplementary Fig.  1c–e ). To prevent ‘gaming’ the study, for each participant a genome from the background was randomly selected, so the 25 ontology terms with the highest outlier scores for the participant could be randomly mixed with the 25 top-scoring ontology terms predicted for the background decoy genome to generate a personalised questionnaire with 50 questions. Each participant was invited to give a binary ‘yes’ or ‘no’ answer to whether they self-identify each phenotype, with the option to leave a comment (Supplementary Fig.  1b ). In a small number of cases participants were recalled and invited to provide information supporting their answers for phenotypes of interest.

The 1KGP3 genomes were used as the background. Before recruitment, the binary yes/no questions were designed with the aim of identifying outlier phenotypes for 5,857 ontology terms, including terms from the Gene Ontology (GO) 26 , Human Phenotype Ontology (HPO) 25 , Disease Ontology (DO) 42 , Medical Subject Headings (MeSH) 43 , and Mammalian Phenotype (MP) 44 ontology databases (e.g. in Table  1 ). See Supplementary Data  3 at https://supfam.org/nomaly for an interactive version and database of ontology term to phenotype question mappings. Participant genome files were processed continuously in batches (eliminating within-batch relatedness) giving an approximately 4–12 hour turnaround between submission of file to the generation of personalized questionnaire; results can be influenced by other genomes in the batch which effectively becomes part of the background for each other. This is because all parts of the similarity matrix interact with each other during the solution of the eigenproblem.

We started with questions designed manually for 5857 different ontology terms. Not all questions were answered but in the end, 94,966 binary self-identified answers were received for 3672 questions, of which 1408 questions received at least one positive answer. Although questions were designed for participants only to self-identify when they are phenotypic outliers, often this was not achieved. To illustrate, a hypothetical bad question is “Do you have myopia?” whereas a good question would be “Do you have myopia worse than -6 dioptres”. Therefore, per-phenotype analysis was carried out for 342 ontology terms whose rate of participants answering ‘yes’ is non-zero and below 5%, and at least a total of 20 answers were recorded per term.

Permutation tests

Statistical evaluation in Fig.  2 was carried out using permutation tests with 100,000 iterations randomly re-allocating the outlier scores, testing the null hypothesis that similar results can be obtained if scores are assigned randomly. For panel b, the average outlier score given to questions that received positive answers (about 0.011) is compared to the averages from random permutations of the dataset. In panel d of Fig.  2 , permuting the scores for each phenotype separately, yields a p -value on the sum of observed scores matching positive answers for each phenotype. All phenotypes can now be ranked by p -value (along the x -axis) for how well the observed scores predict the answers for that phenotype. The observed scores are then randomly permuted and all phenotype p -values recalculated; repeating 100,000 times gives a mean and standard deviation for the number of phenotypes expected by chance at any given p -value. At each point on the x -axis, subtracting the number of phenotypes expected by chance from the observed top phenotypes ( x ) gives the blue line – with confidence intervals. I.e. the number of phenotypes ( y ) likely to be true out of the top x phenotypes ranked by how well outlier scores match the answers.

Data source

Exome data from the DDD 1133 trio sequencing VCF files (accession code: EGAD00001001355 ), 1133 trio family relations, phenotype datasets, validated DNMs ( EGAD00001001413 ), were obtained from the European Genome-phenome Archive at the European Bioinformatics Institute (EGA, https://ega-archive.org/ ).

We ran the 1133 DDD probands, together with 1KGP3 genomes as background, using HPO as the ontology database. Only variants that pass all filters (as specified in the meta-data information from EGA downloads) are included. HPO terms that were used to clinically describe phenotypes in the 1133 trio were mapped to HPO terms in the v1.2 database used in our predictions. Due to the nature of the method, running all of the probands at the same time, in one distance matrix, means that the background is a mixture of 1KGP3 genomes and the other probands. If there was a common genetic cause shared by many probands, it would not get a high outlier score, but we assume enough genetic causes of the disorders are sufficiently independent to be eligible for detection.

For the DDD cohort, evaluation was automated through direct comparison, for each patient, of whether a high-scoring HPO term closely matches a respective clinical annotation. Defining a close match between ontology terms is challenging because the distance over the ontology graph varies wildly in biological meaning; adjacent terms can be very similar or very different. To define closeness, a measure of information content through the graph is needed, for which we used the cumulative number of patients annotated by terms traversing the graph as a metric. The close matches between HPO terms are listed in Supplementary Data  4 . For example we found the following 4 terms sufficiently similar to define as close to each other: HP:0004097 ‘Deviation of finger’ (2 probands), super-term HP:0009484 ‘Deviation of the hand or of fingers of the hand’ (1 proband), and sub-terms HP:0009179 and HP:0004209 – ‘deviation’ and ‘clinodactyly’ of the 5th finger (56 and 2 probands respectively).

It should be noted that the evaluation assumes all potential HPO terms in the DDD database are considered by clinicians, and that an HPO term is not true for the patient if it is not annotated. This represents an underestimation of true phenotypes, but is a necessary assumption for a fair and automatic evaluation.

Comparison to published diagnosis by DDG2P

For patients with true positive predictions, we checked against the list of diagnostic variants in DDG2P from the initial publication 6 and the list of validated DNMs (obtained from EGA) to see if there is a genetic diagnosis and if there is a potential uninterpreted DNM. The 1133 trio DNM list showed that at least one DNM was found for 738 (65%) of children (excluding synonymous, intron, and intergenic DNMs).

HipSci cohort

HipSci exome sequencing data for healthy and diseased people ( EGAD00001003514 , EGAD00001003521 , EGAD00001003522 , EGAD00001003524 , EGAD00001003525 , EGAD00001003516 , EGAD00001003526 , EGAD00001003527 , EGAD00001003517 , EGAD00001003161 , EGAD00001003518 , EGAD00001003519 , EGAD00001003520 ) were obtained from EGA. HipSci open-access exome-sequencing data were downloaded directly from the hosting website ( https://www.hipsci.org/data ).

For each donor, one cell line was selected for the genetics-first prediction according to the following criteria: use primary tissue data if available, otherwise use the iPSC cell line with the minimum changes from the origin cell as measured by number of differences per Mbp, and excluding those where the pluritest or custom CNV check is missing. The result was that 437 cell lines from different donors were selected and the corresponding exome files processed using the 1KGP3 genomes as background, predicting from a set of 5,805 possible GO terms.

There was no confirmation step as with DTC and DDD, so no phenotype data was used. A list of phenotypes with several predicted outliers was examined to find one potentially verifiable by experiment. The list was initially narrowed to four candidates that could be tested on iPS-derived macrophage cells and two candidates that could be tested directly in iPS cells. Expression analysis of genes harboring the variants of interest uncovered a lack of expression for GO:0002741 and GO:1900025 in macrophage, so these were eliminated. The variants implicated in terms GO:0035718 and GO:0002840 are involved in cell signalling (e.g. from the thyroid) and were eliminated as too experimentally complicated. One term, GO:1901223, was excluded due to the lack of availability of a differentiated cell line for a key donor with the variant. Finally, GO:0010826 was selected for experimental validation because five iPS cell lines predicted as outliers were available.

Experimental test for GO:0010826

The Hoik-1 (HPSI0314i-hoik_1), Sehp-2 (HPSI0115i-sehp_2) and Kegd-2 (HPSI0614i-kegd_2) cell lines were selected as control. The Suul-1 (HPSI0514i-suul_1), Yoch-6 (HPSI0215i-yoch_6), Boqx-2 (HPSI0115i-boqx_2), Zapk-3 (HPSI0114i-zapk_3) and Iuoc-2 (HPSI0516i-iuoc_2) cell lines were tested. γ-tubulin was used as a centriole marker 45 . 2.5 × 10 4  cells were plated onto coverslips maintained in 24-well multiwell plates and grown for 2 days. Cells were fixed in cold methanol for 5 min, rinsed, and incubated with 3% (wt/vol) BSA for 1 h. Cells were incubated with γ-tubulin antibody (Sigma-Aldrich, T5326), washed and incubated with the appropriate fluorescent secondary antibody conjugated to Alexa 555 (Invitrogen). DAPI (Thermo Fischer Scientific) was used as counterstain. Cells were mounted in coverslips using ProLong Gold antifade reagent (Thermo Fisher Scientific). Images were acquired with a Nikon A1R confocal microscope. Brightness and contrast were optimised with ImageJ (National Institutes of Health) and Photoshop (Adobe Systems). Quantifications of centrioles were performed manually using ImageJ. Dilutions: the γ-tubulin antibody was used at 1:5000, the secondary antibody at 1:500 and DAPI was used at 1:2000.

Association and confounders

Confounding variables.

Twelve potentially confounding variables were analysed on the DTC cohort data: sex, the first 10 principal components of ancestry, and the genotype array type. Each of the variables was assessed as a predictor on the DTC cohort under the same permutation test as our outlier scores. The correlation coefficients and t-test statistics were also calculated between each variable and the outlier scores for every phenotype. Whilst there is no theoretical reason to believe that predictions from a non-associative method should be confounded by covariates of association, we nevertheless checked this against the above statistics. PERL and packages PDL and Statistics were used as well as R package glm.

Variant associations

There is no expectation that methods based on association will perform well under the same conditions, e.g. high false discovery rate, as the predictor exemplified in this work. However, as a reference point we calculated GWAS statistics on the DTC cohort data using PLINK 46 and subjected them to a 1000 iteration permutation test as in Fig.  3d . Python packages SciPy and NumPy were used here and in other parts of the work. Each phenotype was treated as a cohort with the answers to questions determining case vs control classification. The confounding variables above were used as covariates for the analysis. Within the framework of our reverse approach, predictor scores were used to allocate some of the questions, so to simulate whether the selection of questions contributes to the positive GWAS results, we also repeated the analysis with data points scoring > 0.022 removed. The GWAS results are summarised in Supplementary Fig.  2 . GWAS were performed using a standard approach, as outlined below.

(i) DTC genomes quality check. We started with autosomal, bi-allelic SNPs in the DTC participants that had missingness <1% and those passing an AB ratio binomial test with Z -score <3 for the 1000 genomes project phase 3 data (1KGP3) overall and EUR superpopulation (from GenomePrep). (ii) High-quality common SNPs selection. We restricted variants to having frequency >5% in the 1KGP3, and excluded variants in complex regions from https://genome.sph.umich.edu/wiki/ Regions_of_high_linkage_disequilibrium_(LD) and variants where the ref/alt combinations was CG or AT. We removed all SNPs which were out of Hardy Weinberg Equilibrium (HWE) with a p -value cut-off of pHWE < 1e-8. We LD-pruned using PLINK2 with r2  = 0.1 and 500kb windows. The resulting 22,103 aggregate high-quality sites were used for kinship and genetic components analysis (PCA). (iii) Kinship and ancestry calculation. Data was merged with 1KGP3 and kinship coefficients were calculated among all pairs of samples using PLINK2 and its implementation of the KING robust algorithm. A kinship cutoff of 0.0884 was used to select unrelated individuals (-king-cutoff). Ancestry was inferred via PCA on unrelated 1KGP3 individuals with GCTAv1.93 (plink2) using HQ common SNPs. QC of variants included plink v1.9: HWE deviations exclude p -value <(1e-20), multi-allelic variants were excluded and we filtered variants by missingness <0.02. (iv) GWAS. A total of 4946035 associations were calculated and the multi-phenotype association significance level after multiple hypothesis correction is 1.01 x 10-8. GWAS was run for each phenotype term using ‘Yes’ answers as cases ‘No’ answers as controls. Covariate analysis was implemented with logistic regression of PLINK1.9; sex was imputed (female 1132, male 848, ambiguous 73), and the first 10 PCs from the genetic ancestry analysis were used as well as the version of the genotyping array were used. The permutation plots were generated by running the association on 1000 randomisations of the data and plotted as per Fig.  3d .

Reporting summary

Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Data availability

All data supporting the findings described in the manuscript are available in the article, supplementary information files or from the corresponding author on request. Additionally, supplementary data are made available in an interactive, searchable format via the project webpage at https://supfam.org/nomaly . The DTC cohort data may not be made publicly available because participants are not consented for this, but on application to the corresponding author, requests falling within the constraints of ethical approval granted for the project will be responded to within 14 days. DTC data can be made available under MTA to academic organisations subject to MRC institutional approval and compliance with all relevant data protection laws and requirements. Access is time-limited because we are required to delete all participant data when our work on the DTC cohort ends, which could be before the end of 2023. The database of questions corresponding to 5,857 ontology terms (Supplementary Data  3 ) is also available via the resources webpage (above) and may be a valuable resource for other studies. The similarity mapping, by information content, of all HPO terms that are close to the HPO terms used in clinical annotations by DDD (Supplementary Data  4 ) are also made available on the resources webpage. All data on the resources webpage are also available for download in JSON format. Access to 1KGP3, DDD and HipSci cohorts is only available via those projects directly. Specifically: access to 1KGP3 is made publicly available by the International Genome Sample Resource (IGSR), with data sets accessible from the data portal ( https://www.internationalgenome.org/data-portal ). The DDD cohort data is available from the European Genome-phenome Archive (EGA, https://ega-archive.org/ ), with the study ID EGAS00001000775. To access these data sets, please contact [email protected]. An overview of HipSci cell lines and assay data that are publicly available is available on the cell lines and data browser ( https://www.hipsci.org/data ). It also provides links to publicly available HipSci data in the EBI data archives. To access the managed-access genetic and genomic data in HipSci, please follow the steps stated in the data browser ( https://www.hipsci.org/data#overview ), and apply via the Wellcome Trust Sanger Institute’s Electronic Data Access Mechanism ( https://www.sanger.ac.uk/legal/DAA ).

Code availability

Code for the preparation, parsing and classification of DTC data files is available via the GenomePrep 38 resource already published as an ancillary output of this work that could be of value to others. The website running the pipeline providing questions to participants for the DTC study is available as a free service for up to 500 genotype files per submission (~24hr turnaround), and on request to run on larger datasets; we wish to collaborate with as many genetic cohorts as possible to provide predictions that can be made available within each of their data sharing portals to their registered users. The code used for the phenotype prediction to exemplify the Nomaly framework is not available for download. Individual applications for a license for non-commercial use are possible via the resources webpage ( https://supfam.org/nomaly ) to be considered on a case-by-case basis per project. Pipeline scripts behind the DTC cohort website are available via the resources website on request (not available for direct download due to web security risks). Scripts for conducting the permutation tests, generating the statistics for covariates and producing Fig.  5 are available for direct download via the resources website.

Wray, N. R., Goddard, M. E. & Visscher, P. M. Prediction of individual genetic risk of complex disease. Curr. Opin. Genet. Dev. 18 257–263 (2008).

Manolio, T. A. et al. Finding the missing heritability of complex diseases. Nature vol. 461 , 747–753 (2009).

Visscher, P. M. et al. 10 Years of GWAS Discovery: Biology, Function, and Translation. Am. J. Hum. Genet. 101 , 5–22 (2017).

MacArthur, D. G. et al. Guidelines for investigating causality of sequence variants in human disease. Nature 508 , 469–476 (2014).

Samocha, K. E. et al. A framework for the interpretation of de novo mutation in human disease. Nat. Genet 46 , 944–950 (2014).

Article   CAS   PubMed   PubMed Central   Google Scholar  

The Deciphering Developmental Disorders Study. Large-scale discovery of novel genetic causes of developmental disorders. Nature 519 , 223–228 (2015).

Article   ADS   Google Scholar  

Altman, N. & Krzywinski, M. Testing for rare conditions. Nat. Methods 18 , 224–225 (2021).

Article   CAS   PubMed   Google Scholar  

Tam, V. et al. Benefits and limitations of genome-wide association studies. Nat. Rev. Genet. 20 , 467–484 (2019).

Richards, S. et al. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet. Med. 17 , 405–423 (2015).

Article   PubMed   PubMed Central   Google Scholar  

Minikel, E. V. et al. Evaluating drug targets through human loss-of-function genetic variation. Nature 581 , 459–464 (2020).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Wang, Q., Dhindsa, R.S., Carss, K. et al. Rare variant contribution to human disease in 281,104 UK Biobank exomes. Nature 597 , 527–532 (2021).

Drouin, A. et al. Interpretable genotype-to-phenotype classifiers with performance guarantees. Sci. Rep. 9 , 1–13 (2019).

Davis, J. J. et al. Antimicrobial resistance prediction in PATRIC and RAST. Sci. Rep. 6 , 1–12 (2016).

Article   Google Scholar  

Yu, M. K. et al. Translation of genotype to phenotype by a hierarchy of cell subsystems. Cell Syst. 2 , 77–88 (2016).

Ma, J. et al. Using deep learning to model the hierarchical structure and function of a cell. Nat. Methods 15 , 290–298 (2018).

Grinberg, N. F., Orhobor, O. I. & King, R. D. An evaluation of machine-learning for predicting phenotype: studies in yeast, rice, and wheat. Mach. Learn. 2019 109:2 109 , 251–277 (2019).

MathSciNet   Google Scholar  

Costanzo, M. et al. The Genetic Landscape of a Cell. Science (1979) 327 , 425–431 (2010).

CAS   Google Scholar  

de Lima Morais, D. A. et al. SUPERFAMILY 1.75 including a domain-centric gene ontology method. Nucleic Acids Res 39 , D427–D434 (2011).

Article   PubMed   Google Scholar  

Fang, H. & Gough, J. dcGO: database of domain-centric ontologies on functions, phenotypes, diseases and more. Nucleic Acids Res 41 , D536–D544 (2013).

Fang, H. & Gough, J. A domain-centric solution to functional genomics via dcGO Predictor. BMC Bioinforma. 2013 14:3 14 , S9 (2013).

Google Scholar  

Shi, J. & Malik, J. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22 , 888–905 (2000).

Wright, C. F. et al. Genetic diagnosis of developmental disorders in the DDD study: A scalable analysis of genome-wide research data. Lancet 385 , 1305–1314 (2015).

Wright, C. F. et al. Making new genetic diagnoses with old data: iterative reanalysis and reporting from genome-wide data in 1,133 families with developmental disorders. Genet. Med. 20 , 1216–1223 (2018).

Kilpinen, H. et al. Common genetic variation drives molecular heterogeneity in human iPSCs. Nature 546 , 370–375 (2017).

Köhler, S. et al. The human phenotype ontology in 2021. Nucleic Acids Res. 49 , D1207–D1217 (2021).

Ashburner, M., Ball, C., Blake, J. et al. Gene Ontology: tool for the unification of biology. Nat Genet 25 , 25–29 (2000).

Landrum, M. J. et al. ClinVar: public archive of relationships among sequence variation and human phenotype. Nucleic Acids Res. 42 , D980 (2014).

Amberger, J., Bocchini, C. A., Scott, A. F. & Hamosh, A. McKusick’s online mendelian inheritance in man (OMIM®). Nucleic Acids Res. 37 , D793 (2009).

World Health Organization‎. ICD-10: international statistical classification of diseases and related health problems: tenth revision, 2nd ed. (World Health Organization, 2004).

Auton, A. et al. A global reference for human genetic variation. Nature 526 , 68–74 (2015).

Article   ADS   PubMed   Google Scholar  

Fairley, S., Lowy-Gallego, E., Perry, E. & Flicek, P. The International Genome Sample Resource (IGSR) collection of open human genomic variation resources. Nucleic Acids Res. 48 , D941–D947 (2020).

Mistry, J. et al. Pfam: the protein families database in 2021. Nucleic Acids Res. 49 , D412–D419 (2021).

HA, S. et al. Predicting the functional, molecular, and phenotypic consequences of amino acid substitutions using hidden markov models. Hum. Mutat. 34 , 57–65 (2013).

McLaren, W. et al. The ensembl variant effect predictor. Genome Biol. 17 , 122 (2016).

Zaucha, J. et al. A proteome quality index. Environ. Microbiol. 17 , 4–9 (2015).

Blackford, L.S. et al. An updated set of basic linear algebra subprograms (BLAS). ACM Transactions on Mathematical Software , 28 , 135–151 (2002).

Greshake, B., Bayer, P. E., Rausch, H. & Reda, J. openSNP–A Crowdsourced Web Resource for Personal Genomics. PLoS One 9 , e89204 (2014).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Lu, C., Greshake Tzovaras, B. & Gough, J. A survey of direct-to-consumer genotype data, and quality control tool (GenomePrep) for research. Comput Struct. Biotechnol. J. 19 , 3747–3754 (2021).

Howie, B., Fuchsberger, C., Stephens, M., Marchini, J. & Abecasis, G. R. Fast and accurate genotype imputation in genome-wide association studies through pre-phasing. Nat. Publ. Group 44 , 955–959 (2012).

Danecek, P. et al. The variant call format and VCFtools. Bioinformatics 27 , 2156–2158 (2011).

Li, H. & Barrett, J. A statistical framework for SNP calling, mutation discovery, association mapping and population genetical parameter estimation from sequencing data. Bioinformatics 27 , 2987–2993 (2011).

Schriml, L. M. et al. Disease ontology: a backbone for disease semantic integration. Nucleic Acids Res. 40 , D940–D946 (2012).

Lipscomb, C. E. Medical subject headings (MeSH). Bull. Med Libr Assoc. 88 , 265 (2000).

CAS   PubMed   PubMed Central   Google Scholar  

Smith, C. L. & Eppig, J. T. The mammalian phenotype ontology: Enabling robust annotation and comparative analysis. Wiley Interdiscip. Rev. Syst. Biol. Med 1 , 390–399 (2009).

Moritz, M., Braunfeld, M. B., Sedat, J. W., Alberts, B. & Agard, D. A. Microtubule nucleation by γ-tubulin-containing rings in the centrosome. Nat. 1995 378:6557 378 , 638–640 (1995).

Purcell, S. et al. PLINK: A tool set for whole-genome association and population-based linkage analyses. Am. J. Hum. Genet 81 , 559 (2007).

Waskom, M. L. Seaborn: statistical data visualization. J. Open Source Softw. 6 , 3021 (2021).

Download references

Acknowledgements

We extend our deepest gratitude to all of the DTC participants from numerous countries who made this work possible, generously donating their time and sharing their personal and genetic data for no reward other than the satisfaction of contributing to medical research for the global good. The DDD study presents independent research commissioned by the Health Innovation Challenge Fund [grant number HICF-1009-003], a parallel funding partnership between Wellcome and the Department of Health, and the Wellcome Sanger Institute [grant number WT098051]. The views expressed in this publication are those of the author(s) and not necessarily those of Wellcome or the Department of Health. The study has UK Research Ethics Committee approval (10/H0305/83, granted by the Cambridge South REC, and GEN/284/12 granted by the Republic of Ireland REC). The research team acknowledges the support of the National Institute for Health Research, through the Comprehensive Clinical Research Network. This study makes use of data generated by the HipSci Consortium, funded by the Wellcome Trust and the MRC. We thank Sano Genetics for helping to recruit 557 participants. We thank the LMB visual aids department for help with the figures. We thank Jake Grimmett and Toby Darling for their help with high-performance computing. This work was supported by the Medical Research Council, as part of United Kingdom Research and Innovation (also known as UK Research and Innovation) [MC_UP_1201/14] and a grant from the Biotechnology and Biological Sciences Research Council [BB/N019431/1]. Kings College investigators are supported by the Wellcome Trust and MRC through the Human Induced Pluripotent Stem Cell Initiative [WT098503]. This work was partly supported by the Center for Research and Interdisciplinarity (CRI) Research Fellowship to Bastian Greshake Tzovaras, who also thanks the Bettencourt Schueller Foundation long term partnership.

Author information

Authors and affiliations.

MRC Laboratory of Molecular Biology, Cambridge Biomedical Campus, Francis Crick Avenue, Cambridge, CB2 0QH, UK

Chang Lu, Rihab Gam, Arun Prasad Pandurangan, Himani Tandon, Raju Kalaivani, Minkyung Sung & Julian Gough

Department of Computer Science, University of Bristol, Bristol, BS8 1UB, UK

Jan Zaucha, Hai Fang,  Ben Smithers, Matt E. Oates, Natalie Zelenka, Hashem Shihab, Adam J. Sardar & Julian Gough

Shanghai Institute of Hematology, State Key Laboratory of Medical Genomics, National Research Centre for Translational Medicine at Shanghai, Ruijin Hospital affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China

Centre for Gene Therapy and Regenerative Medicine, King’s College London, Guy’s Hospital, Floor 28, Tower Wing, Great Maze Pond, London, SE1 9RT, UK

Miguel Bernabe-Rubio, James Williams & Davide Danovi

Université de Paris, INSERM U1284, Center for Research and Interdisciplinarity (CRI), Paris, France

Bastian Greshake Tzovoras

You can also search for this author in PubMed   Google Scholar

Contributions

J.G. conceived the DTC study. C.L. and J.G. conceived the studies on DDD and HipSci cohorts, designed and carried out the DTC, DDD and HipSci studies. J.Z., H.F., N.Z. and J.G. developed and implemented the method, with additional contributions to development from C.L., B.S., M.O. and H.S., R.G., C.L., A.P.P., R.K., M.S., A.S, M.O. and J.G. designed the questionnaires. C.L., B.G.T. and J.G.contributed to participants recruitment and correspondence. C.L. and J.G. created and maintained the prediction pipeline, performed data analysis and developed the framework. M.B-R., J.W., and D.D. designed and performed the HipSci centriole experiments. A.P.P., and H.T. performed the structure-based mutational effect analysis. C.L. and J.G. wrote the manuscript. J.Z., H.F. and D.D. contributed significantly to the manuscript. J.G. invented the system, obtained funding, initiated and supervised the project.

Corresponding author

Correspondence to Julian Gough .

Competing interests

Statement: B.G.T. is a co-founder of the OpenSNP database (non-financial competing interest). Patent application (Jan 2016): application number WO2017125778A1; inventors J.G., J.Z. and N.T.; author filed; status is granted in Japan and under examination in other countries; relevant to the method of spectral clustering applied to phenotype prediction. The remaining authors declare no competing interests.

Peer review

Peer review information.

Nature Communications thanks the anonymous reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, description of additional supplementary files, supplementary data 1-5, reporting summary, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Lu, C., Zaucha, J., Gam, R. et al. Hypothesis-free phenotype prediction within a genetics-first framework. Nat Commun 14 , 919 (2023). https://doi.org/10.1038/s41467-023-36634-6

Download citation

Received : 16 February 2022

Accepted : 10 February 2023

Published : 17 February 2023

DOI : https://doi.org/10.1038/s41467-023-36634-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.