2014 · 20 citations · 27 references
ComputingBioinformaticsComputer ScienceParallel AlgorithmsParallel ComputingSequence AnalysisData ParallelismDna SequencingNext-generation SequencingSequence AlignmentDistributed MemoryLarge-scale Data ProcessingNext-generation Sequencing Data AnalysisParallel SequencingParallelizationScalability
Throughput from sequencing instruments has been increasing in an unprecedented speed, leading to an explosion of the next-generation sequencing (NGS) data, and challenges in storing, managing, and analyzing these datasets. Parallelism is the key in handling large-scale data, and some progress has been made in parallelizing important steps, like sequence alignment. However, other major steps continue to be sequential, limiting the ability to handle massive datasets. In this paper, we focus on parallelizing algorithms from two areas. The first is efficient data format conversion among a wide variety of sequence data formats, which is important for cross-utilization of different analysis modules. The second is statistical analysis. Our parallelization sequence data format converter allows sequence datasets in BAM/SAM format to be converted into multiple formats, including SAM/BAM, BED, FASTA, FASTQ, BEDGRAPH, JSON, and YAML, using both shared memory and distributed memory parallelism. The converter currently comprises three instances: SAM format converter, BAM format converter and preprocessing-optimized SAM format converter. Additionally, our converter can also support partial format conversion, to perform format conversion only on a specified chromosome region. The statistical analysis module includes parallelized non-local means (NL-means) algorithm and false discovery rate (FDR) computation. Through extensive evaluation, we demonstrate high scalability of our framework.
27
The Sequence Alignment/Map format and SAMtools
Heng Li, Alec Wysoker, Tim Fennell et al. · Bioinformatics · 2009 · 67.5K citations · Full text
Fast and accurate short read alignment with Burrows–Wheeler transform
Heng Li, Richard Durbin · Bioinformatics · 2009 · 62.8K citations · Full text
BEDTools: a flexible suite of utilities for comparing genomic features
Aaron R. Quinlan, Ira M. Hall · Bioinformatics · 2010 · 30.4K citations · Full text