The LiPP Benchmark Set for Modeling Lipid-Protein Complexes: Comparison of Co-Folding and Docking Methods

Lipid-protein interactions are a universal feature of nearly all cellular pathways and are essential for advances in medicine, pharmaceuticals, and biotechnology. However, computational prediction of lipid-protein binding poses lags behind protein-protein, protein-small molecule, and protein-nucleic acid complexes, partially due to the lack of inclusive data sets of ground truth structures. Here, we introduce the Lipid-Protein Poses (LiPP) benchmark, a curated data set of 311 nonredundant nonannular lipid-protein complex structures that sample a range of protein folds and lipid types. Using LiPP, we systematically evaluate the performance of three AI-based cofolding tools (AlphaFold 3, Chai-1, RoseTTAFold AA), one AI-based docking tool (DiffDock-L), and one physics-based docking tool (AutoDock Vina) to model lipid-protein complexes. Lipid binding poses generated by different software vary in physical plausibility: physics-based docking (AutoDock Vina) and some AI predictors (AlphaFold 3, Chai-1) largely preserve intramolecular and intermolecular constraints, whereas other AI methods (DiffDock-L, RoseTTAFold AA) frequently violate these constraints. AlphaFold 3 demonstrates the highest success rate at 76.1% under the RMSD < 2 Å criterion; however, the success rate drops to 47.2% for structures not seen during training. We identify notable differences in the ability of each software to generate reliable confidence metrics for discriminating accurate from inaccurate lipid binding poses. Our results underscore a substantial need to improve both AI- and physics-based methods for modeling lipid-protein interactions where considerations in lipid size, class, and flexibility are important. The LiPP benchmark set provides a new standardized platform to probe a range of lipid-protein complex modeling tasks. LiPP highlights intrinsic shortcomings in modern docking and cofolding approaches for capturing lipid-protein interaction features and guides the development of next-generation computational tools for advancing lipid biology research.