Enhanced Night-Vision Via a Combination of Poisson Interpolation and Machine Learning

Principal Investigator: Leonard McMillan 
Funding Agency: DARPA 
Agency Number: FA8650-04-2-6543

Abstract

We propose an innovative approach for maintaining the U.S. military's tactical advantage in night-vision for the next several decades. By leveraging abundant computation resources, multispectral and intensified image sensing, and recent advances in image enhancement, we propose to reconstruct night-vision images that rival daytime views. Moreover, in the next 10 years, it might be possible to construct night-vision systems with hyper-acuity, including super-resolution and high-dynamic range. This proposal presents novel approaches to sensing, signal processing, data fusion, and visualization and it addresses new scientific and technical developments to achieve significant increases in existing component performance. We envision a new generation of night-vision systems that provide imagery more easily interpreted by soldiers. Such systems would provide enhanced contrast, improved depth perception, and significantly aid in the accurate interpretation of battlefield situations. Our approach combines the low-level recognition capabilities of domain-specific machine learning with new methods for interpolating images. The degree of enhancement is user controllable, varying all the way from a raw unprocessed night-vision images, to highly augmented daylight approximations. We plan to develop methods based on recent extensions to Poisson equation solvers for handling non-flux conservative initial value problems. These techniques have revolutionized image editing tools (ex. the healing brush in Adobe Photoshop), and their full potential has yet to be explored. An advantage that the image editing problem has over image estimation is the existence of a source image to provide some attributes that are to be transferred to a desired image. In the case of a night-vision, the desired attributes must be derived from the local context and variations in the response multiple synchronized sensors (ex. intensified, FIR, and NIR). It is our belief, that the domain of military applications is suitably constrained (i.e. it is known a priori whether a given contact will take place in jungle or desert terrain), and that extensive sets of training data (daylight example images) could be acquired such that machine learning methods would be sufficient to initialize a Poisson interpolation process. Exploring the feasibility of combining Poisson interpolation along with data-mining and machine learning methods approach is the first stage of this proposed research initiative. Once useful Poisson interpolation methods are developed, it will then become necessary to develop new real-time processing algorithms suitable for video rate processing. Poisson interpolation methods are highly non-linear, and, therefore, efficient processing techniques are still a subject of ongoing research, in contrast to traditional image-processing methods. We propose to explore and hopefully develop efficient algorithms for computing Poisson interpolations as the second-phase of this research program. In a third phase, we plan to incorporate more domain knowledge into the estimation process, by incorporating state-of-the-art target identification techniques geospatial sensor data into the Poisson interpolation process. In a final forth phase we plan explore the possibility of extend night-vision estimation beyond daylight acuity by estimating super-resolution imagery and high-dynamic range capabilities.