We solved the problem of mosaicing scenes with vacant spaces
using the positional information from the IMU available on the quadcopter.
Challenges
The standard stitching methods use feature matching algorithms for
estimating the homography between two images.
Feature matching algorithms require detection of sufficient reliable
features in the images to be matched.
Popular stitching softwares such as Adobe Photoshop are unable to handle vacant spaces.
Scenes containing large regions with vacant spaces result in
very few (or almost zero) features.
At other occasions, there are too many features, when parts of the scene are repeated as seen in the figure.
Contributions
Selection Algorithm: We use the IMU to
select representative images from the video and arrange them into a rectangular grid according to the 'spatial' neighborhood.
It brings down the number of images to be stitched to a manageable number.
It also disambiguates situations when multiple images that are spatially distant, have similar, repeated features.
Whenever there are no features
in the overlapped region of two images, we use the IMU data to find the relative position of one image with respect to another.
Overview
Input imagery is systematically acquired (top left) by a quadcopter. In the next
step, interesting images are found by clustering the video into
regions based on positional data. A graph is constructed using
proximal images. For each connected component in a graph, standard
stitching techniques are used to create mini-panoramas which are
then joined together into a super panorama
again using IMU data.