Integration of the inertial navigation system with consecutive images of a camera by relative position and attitude updating

Abstract

This paper introduces a new method for improving the inertial navigation system errors using information provided by the camera. An unscented Kalman filter is used for integrating the inertial measurement unit data with the features’ constraints extracted from the camera’s image. The constraints, in our approach, comprise epipolar geometry of two consecutive images with more than 65% coverage. Tracking down a known feature in two consecutive images results in emergence of stochastic epipolar constraint. It emerges in the form of an implicit measurement equation of the Kalman filter. Correctly matching features of the two images is necessary for reducing the navigation system errors because they act as external information for the inertial navigation system. A new method has been presented in this study based on the covariance analysis of the matched feature rays’ intersection points on the ground, which sieves the false matched features. Then, the inertial navigation system and matched feature information is integrated through the unscented Kalman filter filter and the states of the vehicle (attitude, position, and velocity) are corrected according to the last image. In this paper, the relative navigation parameters against the absolute one have been corrected. To avoid increasing dimensions of the covariance matrix, sequential updating procedure is used in the measurement equation. The simulation results show good performance of the proposed algorithm, which can be easily utilized for real flights.

Keywords

Inertial navigation system visual navigation image processing feature tracking epipolar constraint integrated navigation system

Get full access to this article

View all access options for this article.

References

Song

Seneviratne

Althoefer

. A Kalman filter-integrated optical flow method for velocity sensing of mobile robots. IEEE/ASME Trans Mechatron 2011; 16: 551–563.

Hérissé

Hamel

Mahony

, et al. A terrain-following control approach for a vtol unmanned aerial vehicle using average optical flow. Autonom Robots 2010; 29: 381–399.

Weiss S, Achtelik MW, Lynen S, et al. Real-time onboard visual-inertial state estimation and self-calibration of mavs in unknown environments. In: 2012 IEEE international conference on robotics and automation, Saint Paul, MN, USA, 14 May 2012, pp.957–964. IEEE.

Panahandeh

Jansson

. Vision-aided inertial navigation based on ground plane feature detection. IEEE/ASME Trans Mechatron 2014; 19: 1206–1215.

Davison AJ. Real-time simultaneous localisation and mapping with a single camera. In: Proceedings of ninth IEEE international conference on computer vision, Nice, 2003, pp.1403–1410.

Jones

Soatto

. Visual-inertial navigation, mapping and localization: a scalable real-time causal approach. Int J Robot Res 2011; 30: 407–430.

Kim

Sukkarieh

. Real-time implementation of airborne inertial-SLAM. Robot Autonom Syst 2007; 55: 62–71.

Weiss S and Siegwart R. Real-time metric state estimation for modular vision-inertial systems. In: 2011 IEEE international conference on robotics and automation, Shanghai, China, 9 May 2011, pp.4531–4537. IEEE.

Jones E, Vedaldi A and Soatto S. Inertial structure from motion with autocalibration. In: Workshop on dynamical vision, October 2007, Vol. 25, p. 11.

10.

Martinelli

. Closed-form solution of visual-inertial structure from motion. Int J Comput Vision 2014; 106: 138–152.

11.

Chiuso

Favaro

Jin

, et al. Structure from motion causally integrated over time. IEEE Trans Pattern Anal Mach Intell 2002; 4: 523–535.

12.

Jones

Soatto

. Visual-inertial navigation, mapping and localization: a scalable real-time causal approach. Int J Robot Res 2011; 30: 407–430.

13.

Soatto

Kosecka

, et al. An invitation to 3-d vision: from images to geometric models, New York: Springer Science & Business Media, 2012.

14.

Webb

Prazenica

Kurdila

, et al. Vision-based state estimation for autonomous micro air vehicles. J Guid Control Dyn 2007; 30: 816–826.

15.

16.

Shen S, Mulgaonkar Y, Michael N, et al. Vision-based state estimation and trajectory control towards high-speed flight with a quadrotor. In: Robotics: science and systems, Vol. 1, 24 June 2013.

17.

Kanade T, Amidi O and Ke Q. Real-time and 3D vision for autonomous small and micro air vehicles. In: 2004 43rd IEEE conference on decision and control (CDC), IEEE Cat. No. 04CH37601, Nassau, Bahamas, 14 December 2004, Vol. 2, pp.1655–1662. IEEE.

18.

Diel DD, DeBitetto P and Teller S. Epipolar constraints for vision-aided inertial navigation. In: 2005 seventh IEEE workshops on applications of computer vision (WACV/MOTION’05), 5 January 2005, Vol. 2, pp.221–228. IEEE.

19.

Mourikis AI and Roumeliotis SI. A multi-state constraint Kalman filter for vision-aided inertial navigation. In: Proceedings 2007 IEEE international conference on robotics and automation, Roma, Italy, 10 April 2007, pp.3565–3572. IEEE.

20.

Hartley

Zisserman

. Multiple view geometry in computer vision, Cambridge: Cambridge University Press, 2003.

21.

Hu JS and Chen MY. A sliding-window visual-IMU odometer based on tri-focal tensor geometry. In: 2014 IEEE international conference on robotics and automation (ICRA), Hong Kong, China, 31 May 2014, pp.3963–3968. IEEE.

22.

Titterton D and Weston JL. Strapdown inertial navigation technology. Reston, VA: AIAA, 2004.

23.

Jekeli

. Inertial navigation systems with geodetic applications, Berlin: Walter de Gruyter, 25 October 2012.

24.

Derenyi EE. Photogrammetry: the concepts. Engineering Lecture Notes No. 57, University of New Brunswick, Canada, 1996.

25.

Van Der Merwe R. Sigma-point Kalman filters for probabilistic inference in dynamic state-space models. Doctoral Dissertation, OGI School of Science & Engineering at OHSU, USA, 2003.

26.

Asl

Pourtakdoust

. UD covariance factorization for unscented Kalman filter using equential measurements update. World Acad Sci Eng Technol 2007; 34: 368–376.

27.

Lowe

. Distinctive image features from scale-invariant keypoints. Int J Comput Vision 2004; 60: 91–110.

28.

Bay H, Tuytelaars T and Van Gool L. Surf: Speeded up robust features. In: European conference on computer vision, Graz, Austria, 7 May 2006, pp.404–417. Berlin, Heidelberg: Springer.

29.

Battiato S, Gallo G, Puglisi G, et al. SIFT features tracking for video stabilization. In: 14th international conference on image analysis and processing (ICIAP 2007), Modena, Italy, 10 September 2007, pp.825–830. IEEE.

30.

Campa G. The aircraft library (Airlib), https://ch.mathworks.com/matlabcentral/fileexchange/3019-airlib/ (accessed March 2004).

31.

VUE. The world generator, https://info.e-onsoftware.com/vue/ (accessed 08 November 2012).