Svetlana Lazebnik

CAREER: Similarity-Based Representation of Large-Scale Image Collections
(NSF Grant No. 0845629)

PI: Svetlana Lazebnik

Students: Joseph Tighe, Yunchao Gong, Megha Pandey, Anson Liang

Collaborators: Jan-Michael Frahm (UNC), Maxim Raginsky (Duke), Florent Perronnin (Xerox Research Centre Europe)

This material is based upon work supported by the National Science Foundation under Grant No. 0845629. Additional funding comes from NSF Grant No. 0916829, Microsoft Research, ARO, Xerox, and DARPA. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

PROJECT GOALS

Intellectual merit: The goal of this project is to develop a representational framework that uses similarity to capture relationships in large-scale image collections. The representation is not restricted to any specific distance function, feature, or learning model. It includes new methods to combine multiple kernels based on different cues, learn fast kernel approximations, and improve indexing efficiency. In addition, new methods for nearest neighbor search and semi-supervised learning are proposed. Two major research problems addressed are: (1) defining and computing similarities between images in vast, expanding, repositories, and representing those similarities in an efficient manner so the right pairs can be retrieved on demand; and (2) developing a system that can learn and predict similarities with sparse supervisory information and constantly evolving data.

Broader impacts: The creation of visual representations and learning algorithms capable of handling large-scale evolving multimodal data has the potential to revolutionize many scientific and consumer applications. Specific application domains include field biology, automatic localization and navigation in indoor and outdoor environments, personalized shopping and travel guides, automated assistants for the visually impaired, security and surveillance.

PUBLICATIONS AND RESOURCES

	Image Parsing Understanding Scenes on Many Levels J. Tighe and S. Lazebnik, To appear at ICCV 2011 SuperParsing: Scalable Nonparametric Image Parsing with Superpixels J. Tighe and S. Lazebnik, ECCV 2010 Project webpage (includes code and data)
	Landmark Photo Collections Building Rome on a Cloudless Day J.-M. Frahm, P. Georgel, D. Gallup, T. Johnson, R. Raguram, C. Wu, Y.-H. Jen, E. Dunn, B. Clipp, S. Lazebnik, and M. Pollefeys, ECCV 2010 Modeling and Recognition of Landmark Image Collections Using Iconic Scene Graphs R. Raguram, C. Wu, J.-M. Frahm, and S. Lazebnik, accepted to IJCV, 2011 Project webpage, data
	Large-Scale Similarity Search and Classification Iterative Quantization: A Procrustean Approach to Learning Binary Codes Y. Gong and S. Lazebnik, CVPR 2011 Project webpage (includes code and data) Comparing Data-Dependent and Data-Independent Embeddings for Classification and Ranking of Internet Images Y. Gong and S. Lazebnik, CVPR 2011 Project webpage (includes code and data) Locality Sensitive Binary Codes from Shift-Invariant Kernels M. Raginsky and S. Lazebnik, NIPS 2009
	Scene Representation Scene Recognition and Weakly Supervised Object Localization with Deformable Part-Based Models M. Pandey and S. Lazebnik, to appear at ICCV 2011