The Backpack Quotient Filter: a dynamic and space-efficient data structure for querying k -mers with abundance. - Institut Pasteur
Pré-Publication, Document De Travail Année : 2024

The Backpack Quotient Filter: a dynamic and space-efficient data structure for querying k -mers with abundance.

Résumé

Genomic data sequencing has become indispensable for elucidating the complexities of biological systems. As databases storing genomic information, such as the European Nucleotide Archive, continue to grow exponentially, efficient solutions for data manipulation are imperative. One fundamental operation that remains challenging is querying these databases to determine the presence or absence of specific sequences and their abundance within datasets. This paper introduces a novel data structure indexing k -mers (substrings of length k ), the Backpack Quotient Filter (BQF), which serves as an alternative to the Counting Quotient Filter (CQF). The BQF offers enhanced space efficiency compared to the CQF while retaining key properties, including abundance information and dynamicity, with a negligible false positive rate, below 10 -5 %. The approach involves a redefinition of how abundance information is handled within the structure, alongside with an independent strategy for space efficiency. We show that the BQF uses 4x less space than the CQF on some of the most complex data to index: sea-water metagenomics sequences. Furthermore, we show that space efficiency increases as the amount of data to be indexed increases, which is in line with the original objective of scaling to ever-larger datasets.
Fichier principal
Vignette du fichier
2024.02.15.580441v1.full.pdf (577.4 Ko) Télécharger le fichier
Licence

Dates et versions

pasteur-04698948 , version 1 (16-09-2024)
pasteur-04698948 , version 2 (18-12-2024)

Licence

Identifiants

Citer

Victor Levallois, Francesco Andreace, Bertrand Le Gal, Yoann Dufresne, Pierre Peterlongo. The Backpack Quotient Filter: a dynamic and space-efficient data structure for querying k -mers with abundance.. 2024. ⟨pasteur-04698948v1⟩
76 Consultations
29 Téléchargements

Altmetric

Partager

More