PipeVal: light-weight extensible tool for file validation
Yash Patel1,2, Arpi Beshlikyan1,2, Madison Jordan1,2,3
1Jonsson Comprehensive Cancer Center, University of California, Los Angeles, Los Angeles, CA 90095, United States.
Motivation:
The volume of biomedical data generated each year is growing exponentially as high-throughput molecular, imaging and mHealth technologies expand. This rise in data volume has contributed to an increasing reliance on and demand for computational methods, and consequently to increased attention to software quality and data integrity.
Results:
To simplify data verification in diverse data-processing pipelines, we created PipeVal, a light-weight, easy-to-use, extensible tool for file validation. It is open-source, easy to integrate with complex workflows, and modularized for extensibility for new file formats. PipeVal can be rapidly inserted into existing methods and pipelines to automatically validate and verify inputs and outputs. This can reduce wasted compute time attributed to file corruption or invalid file paths, and significantly improve the quality of data-intensive software.
Availability And Implementation:
PipeVal is an open-source Python package under the GPLv2 license and it is freely available at https://github.com/uclahs-cds/package-PipeVal. The docker image is available at: https://github.com/uclahs-cds/package-PipeVal/pkgs/container/pipeval.
Related Concept Videos
Data Validation
Nursing assessment guides are generally based on holistic models rather than medical...
P-value
P-value stands for the probability value. P-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme or more extreme as the results obtained from the given sample.
A large P-value calculated from the data indicates to not reject the null hypothesis. But a higher P-value does not mean that the null hypothesis is true. The smaller the P-value, the more...
Detection of Gross Error: The Q Test
Compacting Factor test
The procedure begins by placing concrete into the upper hopper without any compaction. Once filled, the bottom door of this hopper is opened,...
Sign Test for Matched Pairs
To conduct the sign test, we first calculate the differences in...
Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...


