Towards an Embedded Visuo-Inertial Smart Sensor

Abstract

In the neurological system of primates, changes in posture are detected by the central nervous system through a vestibular process. This process, located in the inner ear, coordinates several system outputs to maintain stable balance, visual gaze, and autonomic control in response to changes in posture. Consequently the vestibular data is merged to other sensing data like touch, vision, .... The visuo-inertial merging is crucial for several tasks like navigation, depth estimation, stabilization. This paper proposes a “primate-inspired” sensing hardware, based on a CMOS imaging and an artificial vestibular system. The whole sensor can be considered like a smart embedded sensor where one of the most original aspects of this approach is the use of a System On Chip implemented in a FPGA to manage the whole system. The sensing device is designed around a 4 million pixels CMOS imager and the artificial vestibular set is composed of three linear accelerometers and three gyroscopes. With its structure, the system provides a high degree of versatility and allows the implementation of parallel image and inertial processing algorithms. In order to illustrate the proposed approach, depth estimation with Kalman filtering implementation is carried out.

Keywords

CMOS imager visuo-inertial sensor depth estimation Kalman filter smart sensor SeeMOS

References

Bar-Shalom, Y. and Li, X.-R. (2001). Estimation with Applications to Tracking and Navigation . John Wiley & Sons, Inc., New York.

Benedetti, A. and Perona, P. (1998). Real-time 2-d feature detection on a reconfigurable computer. CVPR '98: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p. 586, Washington, DC, p. 586, IEEE Computer Society.

Benitez, D. and Cabrera, J. (1999). Reactive computer vision system with reconfigurable architecture. ICVS '99: Proceedings of the First International Conference on Computer Vision Systems, London , pp. 348—360.

Böhm, W. , Hammes, J. , Draper, B. , Chawathe, M. , Ross, C. , Rinker, R. , and Najjar, W. (2002). Mapping a single assignment programming language to reconfigurable systems. Journal of Supercomputing, 21(2): 117—130.

Brockett, R.W. (1983). Robotic manipulators and the product of exponentials formula. Mathematical Theory of Networks and Systems: Proceedings of the International Symposium Held at the Ben Gurion University of the Negev, Beer Sheva, June, pp. 120—127.

Chalimbaud, P. , Berry, F. , Marmoiton, F. , and Alizon, S. (2005). Design of a hybrid visuo-inertial smart sensor . ICRA05 Workshop on Integration of Vision and Inertial Sensors (InerVis2005), Barcelona, Spain.

DeHon, A. (2000). The density advantage of configurable computing . Computer, 33(4): 41—49.

Faugeras, O. (1993). Three-Dimensional Computer Vision, a Geometric Viewpoint. MIT Press, Cambridge, MA.

Huster, A. and Rock, S.M. (2001). Relative position estimation for manipulation tasks by fusing vision and inertial measurements. Proceedings of the Oceans 2001 Conference, Honolulu, November, Vol. 2, pp. 1025—1031, MTS /IEEE.

10.

Kalman, R.E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME: Journal of Basic Engineering , 35—45.

11.

Lhuillier, M. and Quan, L. (2002). Quasi-dense reconstruction from image sequence . Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, May, Vol. 2.

12.

Lobo, J. and Dias, J. (2004). Inertial sensed ego-motion for 3d vision. Journal of Robotic Systems, 21(1): 3—12.

13.

Mukai, T. and Ohnishi, N. (1999). The recovery of object shape and camera motion using a sensing system with a video camera and a gyro sensor. Seventh IEEE International Conference on Computer Vision (ICCV 1999), Corfu, Greece, September, pp. 411—417.

14.

Qian, G. , Chellappa, R. , and Zhang, Q. (2001). Robust structure from motion estimation using inertial data. Journal of the Optical Society of America A, 18(12): 2982—2997. Strelow, D. and Singh, S. (2003). Optimal motion estimation from visual and inertial measurements. Proceedings of Workshop on Integration of Vision and Inertial Sensors (INERVIS 2003), Coimbra, Portugal, June.

15.

Woodfill, J. and Von Herzen, B. (1997). Real-time stereo vision on the parts reconfigurable computer. FCCM '97: Proceedings of the 5th IEEE Symposium on FPGA-Based Custom Computing Machines, Washington, DC, p. 201, IEEE Computer Society.

16.

You, S. and Neumann, U. (2001). Fusion of vision and gyro tracking for robust augmented reality registration. VR '01: Proceedings of the Virtual Reality 2001 Conference (VR'01), IEEE Computer Society, p. 71.