Quantifying the human visual exposome with vision language models

The visual environment is a fundamental yet unquantified determinant of mental health. While the concept of the environmental exposome is well-established, current methods rely on coarse geospatial proxies or biased self-reports, failing to capture the first-person visual context of daily life. Here we show that coupling ecological momentary assessment with vision-language models can quantify the semantic richness of human visual experience. Across 2,674 participant-generated photographs, model-derived estimates of "greenness" robustly predicted momentary affect and chronic stress, consistent with established benchmarks. We then developed a semi-autonomous large language model-based pipeline that mined over seven million scientific publications to extract nearly 1,000 environmental features empirically linked to mental health. When applied to real-world imagery, up to 33% of model-extracted context ratings significantly correlated with affect and stress.