Urban-scale facade material mapping from street view images using vision-language models for circular construction...

Area of Science:

• Urban Planning and Design
• Artificial Intelligence
• Computer Vision

Background:

• Reliable building-level data is essential for urban sustainability and circular economy practices but is often lacking.
• Existing data collection methods can be time-consuming, costly, and limited in scope, particularly in data-scarce regions.

Purpose of the Study:

• To introduce URBAN-AI, a novel workflow utilizing multimodal artificial intelligence to extract building material and facade characteristics from street-level imagery.
• To evaluate the accuracy, coverage, and cost-efficiency of the URBAN-AI workflow across diverse urban contexts, from high-income to low-income settings.
• To generate image-based indicators supporting various downstream applications in urban planning and sustainability.

Main Methods:

• Development of the URBAN-AI workflow employing multimodal artificial intelligence for image analysis.
• Application of the workflow across six cities (Zurich, San Francisco, Melbourne, Mumbai, Cape Town, Rio de Janeiro).
• Analysis and verification of outputs using 9,056 high-confidence images, with a focus on facade material classification using vision-language models.

Main Results:

• Achieved a mean composite module accuracy of 87.7% across all evaluated cities.
• Successfully inferred building typology, material type, condition, and architectural style, even in low-income and data-scarce areas.
• Generated valuable image-based indicators for historical facades, retrofitting suitability (seismic, energy), urban morphology, facade greening, and flood exposure.

Conclusions:

• The URBAN-AI workflow demonstrates significant potential for providing essential building data to support urban sustainability and circular economy goals.
• The approach offers a cost-effective and scalable alternative to traditional data collection methods, especially in data-scarce environments.
• The study contributes open resources, including the Global Building Facade Dataset and the URBAN-AI workflow, fostering further research and application.