Adaptive optimization for hybrid vector-relational queries via cost-aware plan switching

In today's analytics and recommendation systems, querying data that combines structured and unstructured forms has become essential. While approximate nearest neighbor search (ANNS) usually converts unstructured data, such as text and videos, to high-dimensional vectors, vector search is implemented as a relational engine plugin in contemporary database systems, which solely rely on ANN indices. Current vector-relational database systems (e.g., CHASE, VBASE) employ fixed execution strategies that commit unconditionally to ANN index traversal for all vector similarity operations, regardless of query characteristics. This design is fundamentally suboptimal: (1) Under highly selective structured predicates (e.g., [Formula: see text]), ANN over-fetches hundreds of candidates of which up to 95% are discarded post-filter, wasting substantial computational resources. (2) Sparse vector distributions cause ANN graph traversal to explore irrelevant regions, degrading efficiency and recall. DYNAMICANN overcomes this with a cost-aware optimizer that evaluates a runtime cost ratio [Formula: see text], where: [Formula: see text] (HNSW graph traversal, distance computations, post-filter overhead); [Formula: see text], [Formula: see text]. The ANN path is selected if [Formula: see text]; scan if [Formula: see text]; else ANN is preferred to preserve recall. We evaluate DYNAMICANN on four representative hybrid query templates (LAION-400M, 512-dim): Q1 (top-K, single filter, [Formula: see text]); Q2 (distance-range, [Formula: see text]); Q3 (join + distance-range, [Formula: see text]); Q4 (windowed top-K, [Formula: see text]). Accuracy is Recall@10: fraction of true nearest neighbors in top-10 vs. brute-force ground truth. By combining adaptive vector search and cost-based relational optimization, DYNAMICANN achieves measured speedups of 2.1-9.8× over CHASE for low-selectivity queries ([Formula: see text]), maintaining Recall@10 [Formula: see text] across all query types, with plan-switching overhead [Formula: see text] of total execution time.