Advancing Generalizable Remote Physiological Measurement Through the Integration of Explicit and Implicit Prior