A knowledge and data-driven optimal planning scheme for multi-modal vision transmission systems

Vision transmission systems (VTS) manages to achieve the optimal information propagation effect given reasonable strategies. How to automatically generate the optimal planning strategies for VTS under specific conditions is always facing challenges. Currently, related research studies have dealt with this problem with assistance of single-modal vision features. However, there are also some other information from different modalities that can make contributions to this issue. Thus, in the paper, we propose a data-driven optimal planning scheme for multimodal VTS. For one thing, the vision features are employed as the basic mechanism foundation for mathematical modeling. For another, the data from other modalities, such as numerical and semantic information, are also introduced to improve robustness for the modeling process. On such basis, optimal planning strategies can be generated, so that proper communication effect can be obtained. Finally, some simulation experiments are conducted on real-world VTS scenes in simulative platforms, and the observed simulation results can well prove efficiency and proactivity of the proposal.