Enabling Fast, Accurate, and Efficient Real-Time Genome Analysis via New Algorithms and Techniques

2025-03-04

AI Summary

High-throughput sequencing—particularly nanopore sequencing—faces challenges including high noise in raw electrical signals, elevated base-calling error rates, and substantial computational overhead, leading to low assembly accuracy, poor scalability, and insufficient real-time performance. To address these, we propose a hash-driven end-to-end analytical framework. Our method introduces RawHash/RawHash2, the first similarity search algorithms designed specifically for raw电信号; proposes BLEND, a noise-robust, variable-length sequence hashing scheme; and develops Rawsamble, the first basecalling-free, all-vs-all overlap tool operating directly on raw signals. By integrating signal feature extraction, hash-based indexing acceleration, and dynamic threshold modeling, our framework achieves nanosecond-scale sequence matching, accelerates real-time analysis by over 3×, and enables—for the first time—the direct *de novo* assembly of raw nanopore signals, achieving both high accuracy and significantly improved computational efficiency.

Technology Category

• Search and Optimization: Evolutionary Computation
• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: The sharing economy
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

The advent of high-throughput sequencing technologies has revolutionized genome analysis by enabling the rapid and cost-effective sequencing of large genomes. Despite these advancements, the increasing complexity and volume of genomic data present significant challenges related to accuracy, scalability, and computational efficiency. These challenges are mainly due to various forms of unwanted and unhandled variations in sequencing data, collectively referred to as noise. In this dissertation, we address these challenges by providing a deep understanding of different types of noise in genomic data and developing techniques to mitigate the impact of noise on genome analysis. First, we introduce BLEND, a noise-tolerant hashing mechanism that quickly identifies both exactly matching and highly similar sequences with arbitrary differences using a single lookup of their hash values. Second, to enable scalable and accurate analysis of noisy raw nanopore signals, we propose RawHash, a novel mechanism that effectively reduces noise in raw nanopore signals and enables accurate, real-time analysis by proposing the first hash-based similarity search technique for raw nanopore signals. Third, we extend the capabilities of RawHash with RawHash2, an improved mechanism that 1) provides a better understanding of noise in raw nanopore signals to reduce it more effectively and 2) improves the robustness of mapping decisions. Fourth, we explore the broader implications and new applications of raw nanopore signal analysis by introducing Rawsamble, the first mechanism for all-vs-all overlapping of raw signals using hash-based search. Rawsamble enables the construction of de novo assemblies directly from raw signals without basecalling, which opens up new directions and uses for raw nanopore signal analysis.

Problem

• Develop noise-tolerant algorithms for accurate genome analysis.
• Enhance real-time analysis of noisy raw nanopore signals.
• Enable de novo assembly from raw signals without basecalling.

Innovation

• BLEND: noise-tolerant hashing for sequence matching
• RawHash: hash-based similarity search for nanopore signals
• Rawsamble: raw signal assembly without basecalling