Where the patterns are: repetition-aware compression for colored de Bruijn graphs⋆

Abstract

AbstractWe describe lossless compressed data structures for thecoloredde Bruijn graph (or, c-dBG). Given a collection of reference sequences, a c-dBG can be essentially regarded as a map fromk-mers to theircolor sets. The color set of ak-mer is the set of all identifiers, orcolors, of the references that contain thek-mer. While these maps find countless applications in computational biology (e.g., basic query, reading mapping, abundance estimation, etc.), their memory usage represents a serious challenge for large-scale sequence indexing. Our solutions leverage on the intrinsic repetitiveness of the color sets when indexing large collections of related genomes. Hence, the described algorithms factorize the color sets into patterns that repeat across the entire collection and represent these patterns once, instead of redundantly replicating their representation as would happen if the sets were encoded as atomic lists of integers. Experimental results across a range of datasets and query workloads show that these representations substantially improve over the space effectiveness of the best previous solutions (sometimes, even dramatically, yielding indexes that are smaller by an order of magnitude). Despite the space reduction, these indexes only moderately impact the efficiency of the queries compared to the fastest indexes.SoftwareThe implementation of the indexes used for all experiments in this work is written inC++17and is available athttps://github.com/jermp/fulgor.