Related Experiment Video
Updated: Mar 3, 2026
09:45
Detection of Copy Number Alterations Using Single Cell Sequencing
Published on: February 17, 2017
12.2K
MarDRe: efficient MapReduce-based removal of duplicate DNA reads in the cloud
Roberto R Expósito1, Jorge Veiga1, Jorge González-Domínguez1
1Grupo de Arquitectura de Computadores, Universidade da Coruña, Campus de A Coruña, A Coruña 15071, Spain.
Bioinformatics (Oxford, England)
|May 6, 2017
Summary
MarDRe is a new tool for removing duplicate DNA sequences from large datasets. It uses cloud computing and Big Data technologies to process data faster than existing methods.
Area of Science:
- Bioinformatics
- Computational Biology
- Genomics
Background:
- High-throughput sequencing generates massive datasets.
- Efficiently processing and cleaning these datasets is crucial for downstream analysis.
- Duplicate and near-duplicate reads can skew results in genomic studies.
Purpose of the Study:
- To introduce MarDRe, a novel de novo tool for duplicate and near-duplicate read removal.
- To leverage Big Data technologies for scalable and efficient data processing.
- To provide a cloud-ready solution for FASTQ/FASTA dataset cleaning.
Main Methods:
- MarDRe utilizes the MapReduce programming model.
- It is built on the Apache Hadoop framework for distributed computing.
- The tool is implemented in Java for cross-platform compatibility.
Main Results:
- MarDRe processes single- and paired-end reads from FASTQ/FASTA files.
- The tool is cloud-ready and designed for cloud-based infrastructures.
- On a 16-node EC2 cluster, MarDRe demonstrated up to 8.52x speed improvement over a state-of-the-art tool.
Conclusions:
- MarDRe offers a fast and scalable solution for duplicate read removal in Big Data genomics.
- Its cloud-native design facilitates deployment on modern computational platforms.
- The tool enhances the efficiency of genomic data preprocessing pipelines.
