Multiscale Gigapixel Video: A Cross Resolution Image Matching andWarping Approach

Abstract

We present a multi-scale camera array to capture and synthesize gigapixel videos in an efficient way. Our acquisition setup contains a reference camera with a short-focus lens to get a large field-of-view video and a number of unstructured long-focus cameras to capture local-view details. Based on this new design, we propose an iterative feature matching and image warping method to independently warp each local-view video to the reference video. The key feature of the proposed algorithm is its robustness to and high accuracy for the huge resolution gap (more than 8X resolution gap between the reference and the local-view videos), camera parallaxes, complex scene appearances and color inconsistency among cameras. Experimental results show that the proposed multi-scale camera array and cross resolution video warping scheme is capable of generating seamless gigapixel video without the need of camera calibration and large overlapping area constraints between the localview cameras.

Introduction

Figure.1: Top: Our capture device. Our system is integrated with one reference camera and 14 local-view cameras. All of them are PointGrey FL3-U3-88S2C-C rolling-shutter cameras without hardware synchronization requirement, and work at 4000X3000 spatial resolution and 15 fps frame rate. In particular, the localview cameras share a focal length fl = 135mm to capture the local high resolution videos, and the focal length of the reference camera is fr = 16mm so as to cover a large field-of-view of an outdoor scene. Each local-view camera can be static or moving during data capture. Bottom: example data captured by our camera array.

Figure.2: Pipeline of our cross resolution matching and warping algorithm. The red arrows denote the feature correspondence building process and the blue arrows denote the warping process.

Results

Figure.3: Zoom in the final composite gigapixel video. (Winter)

Figure.3: Zoom in the final composite gigapixel video. (Summer)

Video

Download