ParaDeco: Frame-Level Parallel Generative Decoding for Large-Scale Surveillance Video Analytics

Video analytics pipelines (VAPs) have been a paradigm for large-scale video analytics. Due to temporal redundancy in video, frame filtering is widely used in VAPs to reduce analysis workload. While prior works have identified decoding as a significant overhead and proposed pre-decoding filtering to mitigate it, we find that codec reference dependencies still force decoders to process many unselected frames, manifesting an over-decoding trap that bounds end-to-end throughput. This limitation stems from the reference-based design in modern codecs, which requires decoding preceding frames to reconstruct any selected one. As a result, over-decoding has become the practical bottleneck in VAPs using modern decoders, highlighting a critical but under-explored problem. To address this issue, we propose ParaDeco, a high-throughput video analytics framework featuring a novel frame-level parallel generative decoder. Unlike traditional decoders, ParaDeco adopts a decode-what-matters approach with decoupled frame dependencies. To decode arbitrary frames independently, ParaDeco generates frame-wise features as standalone skeletons using compressed video metadata, then predicts pseudo frames maintaining semantic consistency with original frames. Moreover, ParaDeco identifies which frames truly matter for analysis via delicate contribution-based frame filtering. We implement ParaDeco on a cloud server and evaluate it on large-scale real-world surveillance video datasets. Our experimental results show that ParaDeco achieves a 2.76× speedup on averagparse e compared to state-of-the-art VAPs.