360 degree stereoscopic image capture makes it easier to capture full scenes in a limited number of images. Because of this quality, it is useful to utilize the these spherical images in computer vision and virtual reality applications such as depth estimation and 3D scene reconstruction. Based on the principles of disparity map generation, we aim to improve 3D stereo reconstruction by using multiple spherical views.
The spherical images are captured by two vertically displaced Ricoh Theta cameras. Each pair of spherical images allows us to generate a disparity map and depth information that can be used for 3D reconstruction. Utilizing multiple viewpoints during scene reconstruction can allow for more robustness when creating translated views. In this project, we discuss our method for improving these depth maps by utilizing multiple spherical views to improve the 3D reconstruction of scenes.
We tested our algorithm on four scenes: A car garage, a living room, a back yard, and a hallway. The car garage was the most successful example, with a good 3D representation of the car and various appliances in the garage. This can be attributed to The living room was not as successful, due to flat surfaces, like walls and patterned objects, like the carpet and curtain. The yard dataset was a special case, providing many cubic shapes and uniform lighting that provided good disparity readings with optical flow estimation. The hallway provided the least accurate results due to barrel distortion of the straight, long walls in the scene. The results of our algorithm on these datasets are shown at the end of the report.