Related Experiment Videos
Building and Validating a Data-Governance Workflow for Artificial Intelligence-Based Tuberculosis Detection from
1Department of Military Science, Military Hospital 175, Ho Chi Minh City, Vietnam. nguyennhatan@gmail.com.
Abstract:
Studies applying artificial intelligence (AI) to tuberculosis (TB) detection from chest radiographs (CXR) focus on models and metrics, yet the quality, provenance and reproducibility of the data workflow decide the reliability of results; duplicate images and non-patient-level splitting inflate evaluation metrics. We propose a seven-step data-governance workflow: multi-source standardisation, provenance management, three-hash deduplication (MD5, pHash, dHash), patient-level splitting, invariant-check auditing, lung segmentation and version freezing by data fingerprint. Four leakage scenarios (deduplication on or off; patient-level or random split) were repeated 20 times with paired splits and compared with the clean scenario on per-repeat differences. On 46,410 CXR images from seven public sources, the workflow removed duplicates at file and content level, standardised 16,554 TB Portals DICOM images, generated 34,554 lung masks without failure and sealed every layer with SHA-256 hashes. On the three-class multi-source corpus, the area under the curve (AUC) saturated near 0.997 in every scenario, consistent with source-class confounding; paired leakage effects were statistically detectable but below 0.1 percentage points. On a single-source probe task (drug resistance within TB Portals), random image-level splitting inflated the internal AUC by 1.7 percentage points (95% confidence interval 0.9 to 2.4; p = 0.001), or 1.2 points after a deduplication gap found during revision was repaired; in a holdout design, an internal gap of 1.1 points shrank to 0.4 points on a patient-disjoint holdout. The magnitude of leakage depends on data structure and must be measured; a rigorous governance workflow makes that measurement, and data reuse, verifiable.