PDCFMO: Probabilistic dense correspondence of human body via fusion meta-optimization

The task of estimating human dense correspondences from images is critical in human-centric analysis, yet existing methods face a trade-off between speed and accuracy. Direct regression approaches are fast but often lack geometric precision, while optimization-based techniques are more accurate but computationally expensive and prone to local minima. This work introduces PDCFMO, a cohesive framework that bridges this gap by harmoniously reconciling the paradigms of broad-scope regression and task-specific meta-optimization. The approach commences with an efficient method for estimating human dense correspondences using a 1D heatmap and a visibility confidence measure, supplemented by a novel technique that generates pseudo-groundtruth visibility using a soft z-buffering ordering scheme, addressing the lack of visibility labels. The key novelty lies in a task-specific neural network-based meta-optimizer that learns descent directions by fusing historical first- and second-order information, integrating task-specific prior knowledge into an iterative optimization process. This improves adaptability to specific settings and handling of complex gestures. Additionally, a memory-efficient Symmetric Rank-one (SR1) inverse Hessian approximation is integrated into the training process, enabling accurate approximations while minimizing memory usage. Evaluations on 3DPW, Human3.6M, and People Snapshot datasets demonstrate notable performance improvements, achieving an eightfold increase in convergence speed over conventional methods, underscoring the framework's robustness and efficiency.