The ever-increasing amount of geocoded images at varying scale, viewpoint, and temporal resolution provides a treasure trove of information for better understanding our environment, to help making better decisions, managing resources, and improve quality of life, particularly in big cities. Geospatial computer vision combines vision and machine learning techniques that scale, to solve real-world problems. In this talk I will present three ongoing projects:
(a) 3D city models: Large-scale semantic 3D reconstruction casts 3D modeling and semantic labeling as a joint problem, where semantic image segmentation enforces class-dependent, geometric priors for reconstruction, while 3D benefits semantic labelling via one joint, convex energy formulation. This leads to more accurate 3D city models that come with category labels directly.
(b) Maps: Semantic segmentation is a fundamental remote sensing task, and most state-of-the-art methods rely on CNNs as their workhorse. A major reason for their success is that deep networks learn to accumulate contextual information over very large windows (receptive fields). However, this success comes at a cost, since the associated loss of effective spa-
tial resolution washes out high-frequency details and leads to blurry object boundaries. We propose to counter this effect by combining semantic segmentation with semantically informed edge detection, thus making class boundaries explicit in the model. We present an end-to-end trainable CNN for semantic segmentation with built-in awareness of semantically meaningful boundaries.
(c) Trees: This project, in collaboration with the Caltech Computational Vision Lab, MIT and USDA/Forest Service, aims to automatically catalogue trees in public space, classify them at species level, estimate their stress level and measure their trunk diameter, to support urban and ecological planning. We propose an automated, image-based system to build up-to-date tree inventories at large scale using publicly available aerial images, panoramas at street-level, and open GIS data of US cities.
[back]
