Bilevel Optimization for ISAC Systems with Proactive Eavesdropping Capabilities

Integrated sensing and communication (ISAC) has attracted extensive attention as a key technology to improve spectrum utilization and system performance for future wireless sensor networks. At the same time, active surveillance, as a legitimate means of surveillance, can improve the success rate of surveillance by sending interference signals to suspicious receivers, which is important for crime prevention and public safety. In this paper, we investigate the joint optimization of performance of both ISAC and active surveillance. Specifically, we formulate a bilevel optimization problem where the upper-level objective aims to maximize the probability of successful eavesdropping while the lower-level objective aims to optimize the localization performance of the radar on suspicious transmitters. By employing the Rayleigh quotient, introducing a decoupling strategy, and adding penalty terms, we propose an algorithm to solve the bilevel problem where the lower-level objective is convex. With the help of the proposed algorithm, we obtain the optimal solution of the analog transmit beamforming matrix and the digital beamforming vector. Performance analysis and discussion of key insights, such as the trade-off between eavesdropping success probability and radar localization accuracy, are also provided. Finally, comprehensive simulation results validate the effectiveness of our proposed algorithm in enhancing both the eavesdropping success probability and the accuracy of radar localization.