The article proposes a new Kalman depth estimation and an improved swarm intelligence optimisation algorithm for adaptive tuning of servo gain for image-based visual servo control. First, a Kalman depth estimation model is established from the principle of image-based visual servoing, and two state equations are designed for depth estimation based on the number of state quantities. Second, the improved sparrow search algorithm is proposed to tune the servo gain adaptively to improve the convergence speed and stability. To verify the effectiveness of the proposed method, the conventional image-based visual servoing and conventional Kalman estimation are reproduced and compared with the proposed method, and the simulation is completed on the Simulink simulation platform for verification. Finally, the experiments are completed in the robotic arm experimental platform. Both the simulation and experimental results show the effectiveness of the proposed method, which reduces the redundancy of the camera and shortens the convergence time.
MilesFAFullerJH. Visual tracking and the primate flocculus. Science1975; 4207: 1000–1002.
4.
WeissLESandersonACNeumanCP. Dynamic visual servo control of robots: an adaptive image-based approach. In: Proceedings of 1985 IEEE international conference on robotics and automation, vol. 2, St. Louis, MO, 25–28 March 1985, pp.662–668. New York: IEEE.
5.
SandersonACWeissLE. Adaptive visual servo control of robots. In: PughA (ed.) Robot vision: international trends in manufacturing technology. Berlin: Springer, 1983, pp.107–116.
6.
EspiauBChaumetteFRivesP. A new approach to visual servoing in robotics. IEEE Trans Robot Automat1992; 3: 313–326.
CorkePRoboticsV. Robotics, vision and control: fundamental algorithms in MATLAB, vol. 73. Berlin: Springer, 2011.
9.
SharmaRSShuklaSBeheraL, et al. Position-based visual servoing of a mobile robot with an automatic extrinsic calibration scheme. Robotica2020; 5: 831–844.
10.
LiLZhangTZhongH, et al. Autonomous removing foreign objects for power transmission line by using a vision-guided unmanned aerial manipulator. J Intell Robot Syst2021; 2: 1–14.
11.
FathianKJinJWeeSG, et al. Camera relative pose estimation for visual servoing using quaternions. Robot Auton Syst2018; 107: 45–62.
12.
TahriOChaumetteF. Point-based and region-based image moments for visual servoing of planar objects. IEEE Trans Robot2005; 6: 1116–1127.
13.
DongJHuYPengK. Robot visual servo control based on fuzzy adaptive PID. In: 2012 international conference on systems and informatics (ICSAI2012), Yantai, China, 19–20 May 2012, pp.1337–1341. New York: IEEE.
14.
LeeSChwaD. Dynamic image-based visual servoing of monocular camera mounted omnidirectional mobile robots considering actuators and target motion via fuzzy integral sliding mode control. IEEE Trans Fuzzy Syst2020; 7: 2068–2076.
15.
RibeiroEGde Queiroz MendesRGrassiVJr. Real-time deep learning approach to visual servo control and grasp detection for autonomous robotic manipulation. Robot Auton Syst2021; 139: 103757.
16.
TongSZhangKShiH, et al. Neural network based visual servo control under the condition of heavy loading. Int J Control Automat Syst2021; 9: 3223–3238.
17.
ShiHWuHXuC, et al. Adaptive image-based visual servoing using reinforcement learning with fuzzy state coding. IEEE Trans Fuzzy Syst2020; 12: 3244–3255.
18.
GhasemiAXieWF. Decoupled image-based visual servoing for robotic manufacturing systems using gain scheduled switch control. In: 2017 international conference on advanced mechatronic systems (ICAMechS), Xiamen, China, 6–9 December 2017, pp.94–99. New York: IEEE.
19.
De LucaAOrioloGRobuffo GiordanoP. Feature depth observation for image-based visual servoing: theory and experiments. Int J Robot Res2008; 10: 1093–1116.
20.
GongyeQChengPDongJ. Image-based visual servoing with depth estimation. Trans Ins Meas Control2022; 9: 1811–1823.
21.
HosodaKAsadaM. Versatile visual servoing without knowledge of true Jacobian. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS’94), vol. 1, Munich, 12–16 September 1994, pp.186–193. New York: IEEE.
22.
ZhongXZhongXPengX. Robots visual servo control with features constraint employing Kalman-neural-network filtering scheme. Neurocomputing2015; 151: 268–277.
23.
WangFSunFZhangJ, et al. Unscented particle filter for online total image Jacobian matrix estimation in robot visual servoing. IEEE Access2019; 7: 92020–92029.
24.
KangMChenHDongJ. Adaptive visual servoing with an uncalibrated camera using extreme learning machine and Q-leaning. Neurocomputing2020; 402: 384–394.
25.
WatanabeKYoshihataYIwataniY, et al. Image-based visual PID control of a micro helicopter using a stationary camera. Adv Robot2008; 2-3: 381–393.
26.
ZhouZZhangRZhuZ. Robust Kalman filtering with long short-term memory for image-based visual servo control. Multimedia Tool Appl2019; 78: 26341–26371.
27.
ZouLWangZHanQL, et al. Moving horizon estimation for networked time-delay systems under Round-Robin protocol. IEEE Trans Automat Control2019; 64: 5191–5198.
28.
LinHSuHShuZ, et al. Optimal estimation in UDP-like networked control systems with intermittent inputs: stability analysis and suboptimal filter design. IEEE Trans Automat Control2015; 61: 1794–1809.
ChakrabortyIMehtaSSDoucetteE, et al. 2.5D visual servo control in the presence of time-varying state and input delays. In: 2017 American control conference (ACC), Seattle, WA, 24–26 May 2017, pp.3741–3746. New York: IEEE.
31.
CaoXZhangQNiZ. Design of hybrid visual servo control system of mechanical arm based on image and position. In: 2017 2nd international conference on robotics and automation engineering (ICRAE), Shanghai, China, 29–31 December 2017, pp.342–346. New York: IEEE.
32.
RastegarpanahAAflakianAStolkinR. Optimized hybrid decoupled visual servoing with supervised learning. Proc IMechE, Part I: J Systems and Control Engineering2022; 2: 338–354.
CaiCSomaniNKnollA. Orthogonal image features for visual servoing of a 6-DOF manipulator with uncalibrated stereo cameras. IEEE Trans Robot2016; 2: 452–461.
35.
SchmidtSF. The Kalman filter – its recognition and development for aerospace applications. J Guid Control1981; 1: 4–7.
36.
XiaolinRHongwenLYuanchunL. Online image Jacobian identification using optimal adaptive robust Kalman filter for uncalibrated visual servoing. In: 2017 2nd Asia-Pacific conference on intelligent robot systems (ACIRS), Wuhan, China, 16–18 June 2017, pp.53–57. New York: IEEE.
37.
XueJShenB. A novel swarm intelligence optimization approach: sparrow search algorithm. Syst Sci Control Eng2020; 1: 22–34.
