HiFormer: Hierarchical Transformer With Box-Packed Positional Encoding for 3D Part Assembly

Estimating the 6-DoF posture of parts in assembly-based modeling is a critical task in the fields of computer graphics, computer vision and robotics. A typical scenario involves enabling a machine agent to automatically assemble IKEA furniture using the provided parts. This paper presents HiFormer, a novel Hierarchical Transformer with Box-packed Positional Encoding, designed for highly automatic 3D part assembly. Our method addresses three important issues commonly encountered in 3D part assembly: 1) How to mitigate the overfitting problem associated with Transformer-based feature learning for 3D point clouds? 2) How to effectively model the relationships between the intragroup and intergroup parts? 3) How to compute positional encoding and integrate it into the Transformer for parts with diverse geometric forms in the coarse-to-fine assembly process? These challenges are tackled through three key contributions: 1) a multi-task 3D Swin Transformer with a two-stage training strategy for feature extraction, 2) a novel hierarchical Transformer for capturing part relationships at flattening, intragroup, and intergroup levels, and 3) an innovative box-packed positional encoding that enhances the Transformer by incorporating query, key, and value information derived from relative box positions. On the PartNet benchmark, our method outperforms the state-of-the-art PWH-MP model on three representative categories-Chair, Table, and Lamp-, achieving average improvements of 2.84% in Part Accuracy (PA) and 3.72% in Connection Accuracy (CA) for diversity modeling (with noise), and 3.55% in PA and 3.21% in CA for deterministic modeling (without noise).