Monocular multi-view stereo imaging system

W. Jiang; M. Shimizu; M. Okutomi

doi:10.2971/jeos.2011.11051

Journal of the European Optical Society - Rapid publications, Vol 6 (2011)

Monocular multi-view stereo imaging system

W. Jiang, M. Shimizu, M. Okutomi

Abstract

In this study, we present a single-camera, multi-view stereo imaging system for capturing three-dimensional (3D) information. First, we design a monocular, multi-view stereo imaging device composed of a fisheye lens, and planar mirrors placed around the lens. The fisheye lens has a wide view-angle. The captured image includes a centered region of direct observation and surrounding regions of mirrored observations. These regions can be considered as images captured by multiple cameras at different positions and orientations. Therefore, the proposed device is equivalent to a synchronous multiple-cameras configuration, in which all the cameras share the same physical characteristics.
In addition, we show how to place the mirrors in order to maximize the common view-angles, which is an important design consideration. Then, after calibrating the projection function of the fisheye lens, we obtain the positions and orientations of the virtual cameras from the external parameters. We also develop two multi-baseline stereo algorithms for the 3D measurement system.
The first algorithm transforms the captured image to perspective images, and uses the traditional method to perform stereo determination. The second algorithm directly uses the original captured image along with an analysis of the epipolar geometry.
Experimental results show that our system is more effective than traditional stereo methods that use a stereo pair, and it can achieve robust 3D reconstruction.

Full Text: PDF

Citation Details

Cite this article

References

E. H. Adelson, and J. Y. A. Wang, "Single lens stereo with a plenoptic camera" IEEE T. Pattern Anal. 14 (2), 99-106 (1992).

O. Avni, T. Baum, G. Katzir, and E. Rivlin, "Recovery of 3D animal motions using cameras and mirrors" Mach. Vision Appl. 21, 879-C888 (2009).

J. P. Barreto, and K. Daniilidis, "Epipolar geometry of central projection systems using Veronese maps" in CVPR'06: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1258-1265 (IEEE Computer Society, New York, 2006).

vision.caltech.edu/bouguetj/calib_doc/index.html

J. Courbon, Y. Mezouar, L. Eck, and P. Martinet, "A generic fisheye camera model for robotic applications" in Proceedings to IEEE/RSJ International Conference on Intelligent Robots and Systems, 1683-1688 (IEEE/RSJ, San Diego, 2007).

J. Diebel, K. Reutersward, S. Thrun, J. Davis, and R. Gupta, "Simultaneous Localization and Mapping with Active Stereo Vision" in Proceedings to 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, 3436-3443 (IEEE/RSJ, Sendai, 2004).

A. R. J. Francois, G. G. Medioni, and R. Waupotitsch, "Mirror symmetry ) 2-view stereo geometry" Image Vision Comput. 21 (2),137-143 (2003).

C. Geyer, and K. Daniilidis, "A unifying theory for central panoramic systems and practical implications" in Proceedings to the European Conference on Computer Vision, 445-461 (ECCV, Dublin, 2000).

J. Gluckman, and S. K. Nayar, "Catadioptric stereo using planar mirrors" Int. J. Comput. Vision 44 (1), 6-79 (2001).

J. Gluckman, and S. K. Nayar, "Rectified Catadioptric Stereo Sensors" IEEE T. Pattern Anal. 24 (2), 224-236 (2002).

A. Goshtasby, and W. A. Gruver, "Design of a single-lens stereo camera system" Pattern Recogn. 26 (6), 923-C937 (1993).

M. D. Grossberg, and S. K. Nayar, "A general imaging model and a method for finding its parameters" in Proceedings to the Eighth International Conference on Computer Vision, 108-115 (IEEE Computer Society, Vancouver, 2001).

S. Hiura, and T. Matsuyama, "Depth measurement by the multifocus camera" in Proceedings to IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 953-959 (IEEE Computer Society, Santa Barbara, 1998).

M. Inaba, T. Hara, and H. Inoue, "A stereo viewer based on a single camera with view-control mechanism" in Proceedings to the International Conference on Robots and Systems,1857-1864 (IEEE, Yokohama, 1993).

H. Kawasaki, Y. Ohsawa, R. Furukawa, and Y. Nakamura, "Dense 3D Reconstruction with an Uncalibrated Active Stereo System" in Proceedings to the 7th Asian Conference on Computer Vision, 882-891 (ACCV, Hyderabad, 2006).

S. Kuthirummal, and S. K. Nayar, "Multiview radial catadioptric imaging for scene capture" ACM T. Graphic. 25 (3), 916-923 (2006).

D.-H. Lee, and I.-S. Kweon, "A novel stereo camera system by a biprism" IEEE T. Robotic. Autom. 16 (5), 528-541 (2000).

C. Mei, and P. Rives, "Single view point omnidirectional camera calibration from planar grids" in Proceedings to 2007 IEEE International Conference on Robotics and Automation, 3945-3950 (IEEE Computer Society, Rome, 2007).

H. Mitsumoto, S. Tamura, K. Okazaki, N. Kajimi, and Y. Fukui, "3-D reconstruction using mirror images based on a plane symmetry recovering method" IEEE T. Pattern Anal. 14 (9), 941-946 (1992).

robots.ox.ac.uk/~cmei/

S. A. Nene, and S. Nayar, "Stereo with Mirrors" in textslProceedings to the Eighth International Conference on Computer Vision, 1087-1094 (ICCV, Bombay, 1998).

M. Okutomi, and T. Kanade, "A multiple-baseline stereo" IEEE T. Pattern Anal. 15 (4), 353-363 (1993).

M. Okutomi, Y. Katayama, and S. Oka, "A simple stereo algorithm to recover precise object boundaries and smooth surfaces" Int. J. Comput. Vision 47, 261-273 (2002).

S. Ramalingam, P. Sturm, and S. Lodha, "Towards complete generic camera calibration" in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1093-1098 (IEEE Computer Society, New York, 2005).

R. Sagawa, N. Kurita, T. Echigo, and Y. Yagi, "Compound catadioptric stereo sensor for omnidirectional object detection" in Proceedings to IEEE/RSJ International Conference on Intelligent Robots and Systems, 2612-2617 (IEEE/RSJ, Sendai, 2004).

M. Shimizu, and M. Okutomi, "Calibration and rectification for reflection stereo" in Preceedings to IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1-8 (IEEE Computer Society, Anchorage, 2008).

J. Weng, P. Cohen, and M. Herniou, "Camera calibration with distortion models and accuracy evaluation" IEEE T. Pattern Anal. 14, 965-980 (1992).

H. H. P. Wua, M. T. Lee, P. K. Weng, and S. L. Chen, "Epipolar geometry of catadioptric stereo systems with planar mirrors" Image Vision Comput. 27, 1047-1061 (2009).