Positioning-Based Photo Retrieval

MMAI 2015 FINAL PROJECT

 

To Know Where We Are: Positioning-based Photo Retrieval

 

2015/12/16 Update

To Know Where We Are: Positioning-based Photo Retrieval

Input: query photo

Return: other photos taken in the same position with different direction.

Method: use training photos collected from Internet or somewhere else to bulit a 3D model, perform 2D-3D matching when the query photo comes. As a result we get the position where the query photo was taken, then we use the position to final all photos taken in the same position or those was taken very close to this position ------- A new concept of image retrieval.

Topic 1    Photo Tourism In Campus (existing)

Use an amout of photos to built a campus model in 3D, and enable the guest to interactively move about the 3D space by seamlessly transitioning between photographs.

Method: 

Snavely, et al. "Photo tourism: exploring photo collections in 3D." ACM transactions on
graphics, 2006.

http://phototour.cs.washington.edu

Topic 2    Best Shooting Point Seeking

Use an extension of the above work to find the best position to take a photo in a scene. This could be achieved by finding the viewpoints-densest area in the 3D model.

Topic 3    Photo-based Positioning

An extension of the above work. By using our own photo to quey the 3D model in a feature-based method, we could find where we are as well as get the photos taken near our current position.

Method:

Sattler, et al. "Fast image-based localization using direct 2D-to-3D matching." ICCV 2011.

Topic 4 CBIR System based on pretrained model feature extraction

After HW2, I have tried features extracted by pretrained model of GoogleNet. It turned out that such features was effective and thus led to more than 80% presicion(MAP) on our database.