Statistical Degradation State Modeling for Mask-Free Shadow Removal

Shadow removal is fundamentally an ill-posed physical inverse problem, often complicated by continuous photometric gradients (penumbras) and their complex entanglement with high-frequency background textures. Existing paradigms, relying either on explicit spatial masks or implicit semantic priors from massive vision models, face challenges in reliably quantifying this continuous local attenuation, leading to structural artifacts or incurring substantial computational costs. To address these limitations, we argue that shadows should be modeled as fine-grained, physics-guided degradation states rather than heuristic spatial regions. In this paper, we propose a shadow degradation state representation paradigm by mapping the classic physical model into the second-order statistical space of the Luminance-Variation (LV) domain. Within this domain, we geometrically reveal the non-orthogonal coupling between intrinsic texture dispersion and physical illumination attenuation. To mathematically operationalize this insight, we introduce an oblique decoupling mechanism, instantiated by a structured generative estimator, that robustly extracts the baseline attenuation and effectively isolates texture fluctuations from photometric uncertainty. Building upon this, we embed the state representation into a deep-unrolled Generalized Expectation-Maximization (GEM) architecture, EM-LVNet. By alternating between statistical prior inference and uncertainty-guided spatial restoration, this framework establishes a theoretically grounded optimization loop. Extensive evaluations demonstrate that EM-LVNet achieves state-of-the-art mask-free restoration with compact computational overhead. Crucially, it exhibits superior cross-dataset generalization and strong empirical inference idempotence, validating the stability and robustness of our physics-guided degradation modeling. https://github.com/XYLGroup/EM-LVNet.