Virtual screening and experimental validation of small-molecule compounds targeting AR in prostate cancer

Prostate cancer ranks as the second most prevalent malignancy among men, with its progression predominantly driven by androgen receptor (AR) signaling. Despite the centrality of androgen deprivation therapy (ADT) in managing advanced prostate cancer, the emergence of resistance culminating in castration-resistant prostate cancer (CRPC) remains a formidable challenge. In this study, an integrative strategy for virtual screening was developed using a machine learning-based model implemented with Random Forest, followed by molecular docking. This strategy was employed to screen approximately 1,500,000 compounds, ultimately narrowing them down to 20 candidates. Among these, 8020-1599 and C301-6562 were identified as effective AR inhibitors. In vitro assays demonstrated that these compounds significantly inhibited the proliferation, migration, and invasion of prostate cancer cells, exhibiting efficacy comparable to that of the clinical standard, enzalutamide. In vivo experiments further validated their antitumor activity, demonstrating significant tumor growth inhibition without causing notable toxicity. Mechanistically, 8020-1599 and C301-6562 disrupted AR nuclear translocation and its downstream signaling pathways, leading to a marked reduction in the expression of AR-regulated genes FKBP5 and KLK3. This study highlights a promising approach for developing highly effective and minimally toxic AR inhibitors, although further research is required to assess their long-term safety and potential effects on alternative signaling pathways.