Double-head transformer neural network for molecular property prediction

Area of Science:

• Computational chemistry
• Machine learning
• Deep learning for drug discovery

Background:

• Current deep learning models for molecular property prediction struggle with feature generalization and weight assignment.
• This limits the achievable accuracy in predicting molecular properties.

Purpose of the Study:

• To develop a high-precision molecular property prediction model.
• To address limitations in generalization and feature weighting in existing deep learning methods.

Main Methods:

• Proposed an end-to-end double-head transformer neural network (DHTNN).
• Introduced a novel activation function, 'beaf', to improve nonlinear feature representation generalization.
• Incorporated a residual network for stable model convergence and addressed gradient explosion.
• Utilized double-head attention to extract intrinsic molecular features and assign weights.

Main Results:

• The DHTNN model demonstrated significant performance improvements on the MoleculeNet benchmark dataset.
• Achieved higher accuracy in molecular property prediction compared to state-of-the-art methods.
• The 'beaf' activation function enhanced the generalization of molecular feature representations.

Conclusions:

• The proposed DHTNN model offers a superior approach for high-precision molecular property prediction.
• The integration of specific architectural components and a novel activation function overcomes key limitations in deep learning for cheminformatics.
• DHTNN represents a significant advancement in accurately predicting molecular properties.