Abstract
Robot can help human in their everyday life and routine. These are not an indoor robot which was designed to perform desired task, but they can adapt to our environment by themselves and to learn from their own experiences. In this research we focus on high degree of autonomy, which is a must for social robots. For training purpose autonomous exploration and unknown environments is used along with proper algorithm so that robot can adapt to unknown environments. For testing purpose, simulation is carried with sensor fusion method, so that real world noise can be reduced and accuracy can be increased. This dissertation focuses on the intelligent robot control in autonomous navigation tasks and investigates the robot learning in following aspects. This method is based on human instinct of imitation. In this standard real time data set is provided to the robot for training purpose, it gets train from these data and generalize over all unseen potential situations and environments. Convolutional Neural Network is used to determine the probability and based on that robot can act. After acceptable number of demonstrations, robot can predict output with high accuracy and hence can acquire the independent navigation skills. State-of-the-art reinforcement learning techniques is used to train the robot via interaction with the robots. Convolutional Neural Network is also incorporated for fast generalization. Robot is train based on all past state-action pairs collected during interaction. This training model can predict output which helps robot for autonomous navigation.
Keywords
Introduction
Autonomous navigation is rapidly being utilized in many research. This can include the applications like transportation, security, automation, Robotics and entertainment. Robotics is widely using autonomous navigation system, where it has a potential to operate the robot using navigation and hence robot can navigate into physical world in normal manner as human do. Robotics includes robots which can be further classified as indoor and outdoor robots [2]. But in both of the above mentioned cases robots has to navigate into scene of the real world and this physical location can be changing or can become complex or simple with respect to time. In this navigation many other electronics tools can be used which includes sensors and high end processors like GPU, so the robot can perceive and provide knowledge from the scene.
Navigation task can be organized as 2 step scenario. In the first one it involves the problem of perceiving tin environment, in which robot creates maps for navigation. This problem is referred as mapping, in second step robot localizing in the given scene, so that orientation can be optimized and navigation can be carried out. GPS stands for global positioning system is found out to be the best solution for navigation for many travelers, but in case robots it can’t solve the issue of navigation specially in case of indoor navigation. Vision based and Inertial navigation has best promising substitutes [3]. In case of vision-based indoor navigation many researches are going on the main cause of this ongoing research is easy availability of cameras and other sensors required for high dimensional images, this can revolved the vision-based indoor navigation. Along with that there is lot of advancement t into the computing information from the single camera, this can involve the obstacle detection technique, 3D modeling ad tracking. All this techniques has a potential to put robot navigation to the next level. Because of drawbacks which we mentioned above, in this paper end to end trainable Siamese DCNN is used to navigate the robot within the indoor scenes.
Many researches have been carried out on vision-based autonomous navigations based on Time of Flight (ToF) principle and it uses the finding range method for indoor navigation [4]. This researches are carried out from decade and it has a strong impact with respect to state-of-art. But in the ToF based methods also has some limitation like in the due to propagation of visible light it struggle to get artefacts and depth holes. Stereo vision also now a days uses scene reconstruction and mapping technique. This method can be alternately used for ToF-based approaches [7]. This method generally usages image pairs, and hence because of it creates the problem of weak textural features. SLAM stands Simultaneous Localization and Mapping which is based on principles of stereo vision and ToF has evolved the state of arts in recent time [8]. SLAM techniques has high accuracy and it is dependable hence many researches has been carried out, it is observed that even it has high accuracy, it is not practically possible to implement it because of high use of energy, for many mobile robots such as Automated Guided Vehicles (AGV) and Unmanned Aerial Vehicles (UAV), in real world it is not possible. Another issue with SLAM is that it inherit some properties from stereo vision and ToF because of that it creates some problem in indoor environment this involves significant occlusion, scale variation and textural features [9]. In this paper this drawbacks are studied and based on that Siamese DCNN is developed to handle the indoor autonomous navigation.
The proposed technique of deep Siamese solves the issues of scene mapping and self-orientations via sematic scene segmentation branches and scene classification. significant number of convolutions layers are put into two branches which increases the efficiency in term of inference latency and memory.
The dataset which is designed for indoor navigation is used for training the model to convergence and the performance is verified though live experimental scenario and simulation. It is found that the proposed system performance very well with respect to robustness, real-time performance and accuracy without the need of stringent off-line calibrations and additional hardware. This paper is organized as follows.
Background
One of the fundamental requirement of mobile robot is navigation, this navigation can be indoor or outdoor. Main application of robot involves cleaning, rescue, and navigation and so on. For this it is very much important that robot should be mobile and hence it is should effective is obstacle detection and avoidance, from both known and unknown environment. In many previous researches probabilistic models are used, in which cost maps are calculates, in the computation training and testing is carried out on known environment, so that it become difficult to reliable on such algorithm in case of real world. In such problem data is provided to the roots and dealing with unknown scene becomes difficult, fortunately with deep learning many layers of knowledge extraction can be computed, which provides the potential solution to the such problem.
Motivation
Deep neural networks, was firstly proposed by Fukushima in 1980 it was developed from artificial neural network. From last decade there is a lot of advancement tin the field of deep neural networks it has been evolved and it application is present almost in every field which is natural language processing, computer vision, and now robotics is also using deep learning. Along with this there are many more application of Deep neural networks In deep learning multiple processing layers are used and information or hidden patterns are found out on this various layers. It derived from the ANN, which was developed by McCulloch and Pitts in the 1940 s. Structure of Artificial neural network is much similar to brain, in which neurons are present to transmit the information via neural structure. In ANN and deep learning similar approach is used, different layers of knowledge extraction is created which can extract the information and detect hidden pattern present in the data which then can be utilized to make decision.
Aim of the research
Aim of this research is to control a mobile robot to work in an internal environment (indoor) through the use of deep learning technology (DL) and deep reinforcement learning (DRL) algorithms that is using python program, Graphics Processing Unit (GPU) and TensorFlow library. This training model help control mobile robot to detect object and avoid obstacle. By this training the robot learning from during set images in dataset(such as A12 THOR) and test it to output direct control commands robot Movement such as (left, right, forward, backward), after that we enable the real mobile robot to Movement and avoid obstacles use learning model by images from the camera provide on robot.
Related work
Obstacle avoidance through deep networks based on intermediate perception
In this research, a new method to predict a trajectory from images is proposed. NYUv2 dataset is used to train the model. Figure 1 shows structure of network. Designed cost functions automatically calculates the ground truth trajectory. The CNN perception is spitted into two phases:
Structure of network.
This model works perfectly on other indoor databases.
Using CNN, trajectories are predicted directly from one single picture. Trajectories contain more 3D information than instant steering command. For trajectories perdition it is necessary to get ground truth labels from RGB-D images automatically without human demonstrations. After this a two-stage CNN with intermediate perception is applied. There are 2 phases in this algorithm.
Phase 1: In phase 1 depth and normal surface from an image is predicted, this are two important geometric properties related to 3D obstacle representation.
Phase 2: In this phase normal maps and depth are predicted using another Convolutional Neural Network model.
Input to the model is an Red Green Blue image. intermediate result and other direct result are shown in the last two columns in red colour. From right to left: predicted paths, predicted surface normal, predicted depth picture and Red Green Blue image. From Fig. 2, it can be observed that proposed model and RGB prediction is similar, in the later stage this model result is used to predict output with high accuracy.
Example of path prediction on NYUv2 dataset (Yang, 2017).
Instead of human demonstration, designed 3D cost function is used to determine the ground truth trajectory and its label. Mediated perception technique is used to train Convolutional Neural Network model so that it can determine surface normal and learn depth before path prediction. Robot simulation technique is applied so that robot can avoid different random set of obstacles.
Monocular vision based obstacle avoidance
In this model, robot interacts with environments with a monocular camera for decision making using this robot can avoid obstacle in the way. In this research, a duelling architecture based deep double-Q network (D3QN) is proposed for obstacle detection snd avoidance, which uses only monocular Red Green Blue vision. Based on this technique, D3QN can predict the obstacle with high accuracy even when the dataset are noisy. It is observed that it enables the two-fold acceleration which improves the accuracy as the training is carried out, its learning ability is faster than normal deep Q models, so that very shortly it can be transferred to real robots. It has very well accuracy with generalization to the unknown objects.
An RGB is provided as an input to the system, depth prediction model which includes convolution residual network. This fully convolutional neural network can predict the depth from the RGB image, output of this layer is depth image stack. After that a deep Q network is applied which consists of a convolutional Neural network and a duelling network, because of that It can parallel predict the Q values for linear action as well as angular actions.
Fig. 4 shows the training procedure of D3QN network, in this real image is provided as a dataset, in this xt is a current state, an action is carried out, r stands for reward and Xt+1 is the resulting state. In this back-propagation technique is used. In back propagation the total loss is propagated toward network so that the weight at each node is adjusted in such a way that total loss can be minimized, using this technique accuracy of the algorithm is increased. ⊕ is stand for addition and ⊗ denotes multiplication. Simulator is used for training purpose so that robot can make decision in various environments. Deep reinforcement learning found to be very impressive in recent years. It usually uses weak learning signals and hence it requires large amount of dataset to train the model. This is the reason it is difficult to use this technique into robots. Therefore, simulations which have a failure and recovery mechanism is used. In this paper, we focus on the problem of obstacle avoidance with monocular camera. Figure 3 shows the Monocular Vision based Obstacle Avoidance. This paper mainly focuses on the below findings: Monocular vision-based obstacle avoidance technique is adopted with high accuracy and throughput. D3QN is found to increase the speed of obstacle detection and avoidance, even when the computational resources are less. The hidden pattern discovered from simulation can be easily transferred to real world scenarios. Real world dataset and environment is used to do experiment analysis to find out the performance of the model.
Monocular Vision based Obstacle Avoidance (Xie, Markham, 2017).
Training procedure of D3QN (Xie, Markham, 2017).
There are two main issues in deep learning in mobile robot, these are Lack of generalization ability to changing environment Data inefficiency
It is observed that deep learning mobile robot model, need lot of data to train and again it is very costly as there is need to many trial and error, so that system can learn, as system grows it gives better result, but it is not always possible another problem in this approach is it fails in different environment, as new dataset is unknown it becomes very difficult to compute new output.
These issues can be handled by making advancement in target-driven visual navigation. Which is shown in the Fig. 5. To address the issue of lack of generalization, actor critical model is proposed. It maintains current state and output which help in generalization. Data inefficiency is handled by AI2THOR framework. Using AI2THOR framework, physics engine and high quality of 3D scenes were captured. These enable to take action and so that it can interact with objects. Using this approach huge amount of training sample is collected. It converges faster than state-of-the-art model. It can be generalized across different scenes and targets. Model is trained on simulation which helps to deal with real time case scenario. This model doesn’t require feature engineering.
Visual Navigation based on target (Y Zhu, 2016).
In this research visual input is used to navigate the robot. For better navigation technique it is required to improve actions according to environment.
Basic principle of this model is to take capture image and environment as an input and then process the data and find the path using algorithm without re-training. Using above image, we can observe that image is taken and generic siamense layers produces the scene specific layer which helps robot to act according to input. These techniques don’t need prior map of environment; this is the main advantage of this model. Limitation of this model is it doesn’t re-train, but when DRL is applied to multiple targets, network should learn from it and should be re-trained again. Lack of re-trained cause less generalization to the changing environment. That’s why to increase the accuracy of the model is it required to re-train again. Figure 6 shows that the deep siamese actor-critic model.
Deep siamese actor-critic model (Y Zhu, 2016).
Fig. 1 presents convolution neural network which address the complex vision tasks like image recognition and visual Object tracking (VOT) [2]. To control a mobile robot to work in an internal environment(indoor) through the use of deep learning technology (DL) and deep reinforcement learning (DRL) algorithms that is using python program, Graphics Processing Unit (GPU) and TensorFlow library. This training model help control mobile robot to detect object and avoid obstacle. By this training the robot learning from during set images in dataset(such as A12 THOR) and test it to output direct control commands robot Movement such as (left, right, forward, backward), after that we enable the real mobile robot to Movement and avoid obstacles use learning model by images from the camera provide on robot.
The proposed model is designed in such a way that it can share maximum convolutional operations as possible, so that it can increase the efficiency with creating any consensuses to the prediction of accuracy of both branches. This produces the model input as single and other shared convolutional blocks which divides into semantic scene segmentation branch and scene classification branch. In that a shared convolutions block is made up of three max pooling layers and ten convolutional layers. First the input image is of 3px*512px*512px which is then converted to 512 * 64 *64 activation map. In DCNN there is a iterative combination of down-sampling and polling approach as observed in shared convolutional block, which enhance the sparse spatial resolutions. This is a contrast to the sematic based DCNN which uses denser feature maps to maintain input-level resolution. This is the main reason for splitting two branches after 10th convolutions in shared convolution block. Below the design detail of two branches are present. The importance of scene classification branch to calculate the angle and position of robot from the other scene which is captured. Outcome of this stage is used to optimize the global position and orientation with respect to scene. Therefore scene convolution branch is made up of 3 convolutional layers, in which there are two global average pooling layer and two max pooling layers. In between two global average pooling layer one “Dropout” Layer is placed so that it can increase the regularization of performance. Finally the complete layers connects the final branch to Softmax layer which results preselected control strategy. In this paper six different control strategy for Softmax layers are decided based on the human pilot trails. Figure 7 shows that the Navigate and find objects in a virtual environment.
Navigate and find objects in a virtual environment (images in dataset such as A12 THOR).
These control strategy of Softmax helps to adjust the position and orientation of robot with respect to scene captured from environment as follow. At each instance the θ value is measured by the offset of the current orientation and optimal orientation, θ can be assumed as set (0°,±45°,±90°,180°). To extract the path from the pixel wise segmentation of incoming scene the semantic scene segmentation branch is prepared. The important requirement for this branch is to efficient reconstruction of less resolution feature into dense multidimensional feature so that it can achieve the scale invariant and accurate maps of the scene. However the aim must be such that receptive filed should not get sacrificed. To deal with this problem Atrous Convolution is used in DCNN which creates the consecutive pooling operation modern. This idea is extended into ASPP - Atrous Spatial Pyramid Pooling, it can efficiently works with multi-scale feature into effective receptive field of the model. ASPP captures the images are 3 different scales with different dilation rates of 18, 12, 6. In this research ASPP is modified as follow: Firstly ASPP in algorithm uses 5 parallel convolutional branches, we removed the image feature branch so that it cuts down the feature depth to 32px*4px from 32px*5px, and only 4 parallel branches. Secondly we minimize the count of filters in each convolutional branches to 32px from 256px, which prune the intermediate feature size to 32px*4px from 256px*4px. Result of this step production of feature with small magnitude than normal. The newly formed reduced featured depth is optimized for the accurate segmentation for indoor navigation. In this there are only 2 pixel categories are present which are reachable path and obstacles. Figure 8 Shows the Control strategies.
Control strategies.
On the other hand, this model reduces the number of features which increases the efficiency of the model. The parallel branches are stacked with each other and followed by feature refining convolutional layers, “Conv1” and “Conv2”. After this level “unpool” layer is applied for restoring the feature maps to input-level resolution. For efficient rebuilding input resolution of feature maps, bipolar interpolation is used. The importance of this interpolation scheme and other sampled points has been highlighted in other researches [27]. In this branch, softmax layer is connected for pixel wise distinction into 2 categories.
Dataset: The House Of interaction (A12 THOR) data set is used for experimental purpose. It is very difficult and unreliable to acquire large-scale action and interaction data of real environments. The House Of interaction (A12 THOR) data set acts like a simulation framework. Using this framework large number of large numbers of visual observation for different action and reaction is considered. This paper represented the training of both of task to converge. Training technique is described below. Firstly the shared block is trained on the A12 THOR stands for House Of interaction dataset [12] with this scene classification branches are also present. This is because A12 THOR is extensively designed for autonomous navigation, so that model can learn weight which represents the mid and low level saliency image features which includes blobs, shapes and edge information. These feature are useful for images.
Because A12 THOR dataset is not labeled for sematic segmentation the semantic scene branch remains frozen. IN the second phase the CityScapes datasets are transfer with segmentation branch alone [2]. Even though CityScapes are designed for the outdoor navigation it uses the significant number of images so that it can be use in indoor communication. CityScapes are restructured in the way that it can class the five object into perpendicular to skyline and visually flat, this five classes are road, ground, sidewalk, rail track and parking, it is converted to obstacle and reachable path.
After this stage fine tuning is carried out,. In the Fig. 9 it is explained very well in which RGB camera are present on the robot and an human guide navigate through indoor scene, this environment is recorded in the form of video for twenty minutes each. After that the image is sampled to frame for production of picture dataset. Semantic labeling is carried out on the dataset, so that images can be grouped into category.
Indoor scene (Yao Yeboah, 2018).
Model training
Two different variant of Siamese DCNN model is implemented. In both of this model the component presentation is same, but only one difference is, first variant is motivated form the VGG-16 [4], while the second variant is adopted from residual framework of ResNet50 model [9]. It is noted that this two model variant has different depth within semantic scene segmentation branches and scene classification, while the resolution and feature depth in scene classification remains the same.
These model are implemented in python using library Keras and tensor flow as backend. Along with that cuDNN v5.1 and CUDA v8 is used for calculation of acceleration during the training. for inference on input image within 12 millisecond, the robot is provided with single GeForce GTX 1080 GPU card, in the testing phase. Scene classification and convolutional block is first initialized with A12 THOR weights so that it can reduce the overall training time. Then semantic segmentation Brach is trained on CityScapes by using Adam optimization, with the batch size of 16 and the learning rate of 0.003. Time required for completion of this stage is almost 9 hours. After that Fine-tuning is carried out on each branch till it converge, and the speed of knowledge extraction is 1e-5 and SGD (Stochastic Gradient Descent) with 100 epochs. Due to the issue of data overfitting because of textural similarities within the indoor scene, batch normalization technique is used to introduce large variance into model activations learned during the training, which allows the weights to be generalized. In this research the some adoption is performed which are rescaling at 1./255 and shear range of 0.2.
Results & discussion
In general to calculate the performance of model different approaches are used, this may include F 1Metrics, Accuracy, Recall and Precision, in case of multi-category scenarios, it is necessary to consider all this aspects. In this model comparison macro form of this metrics is used. Along with that for analyze the performance of this two variant, confusion matrices are prepared. In the experiment 480 random scene are selected for the testing purpose. It is observed that both the model variant provide good analysis result, the comparison metrics for variant 1 outperform variant 2. This is because of their intrinsic structural variations, for variant 1 it is VGG based, while for Variant 2 it is ResNet based. In the first variant the abstraction of layer is carried out which is received from successive layers, it makes them high effect for transfer learning phase. While in case of ResNet- based architectures only refinement is carried out on successive layers. . The confusion matrix of the proposed system is defined in Fig. 10.
Confusion matrices achieved by the two model variants.
Extracted features before and after fine tuning.
However, both of the variant of this model shows promising performance for classification task with accuracy rate more than 99% Quantitative semantic segmentation n result is observed by model variant after branch fine tuning and before branch fine tuning, this is represented in Table 2. From Table 2 it is observed that this model efficiently allows the initialization. It is also observed that because of fine-tuning, performance of the model improved significantly.
Branch performances
Semantic segmentation results measured with IoU for both model variants before and after fine-tuning
The impact of fine tuning is analyzed in model variant after branch fine tuning and before branch fine tuning, The features represents the grouping in alignment and cleat refinement with categories after the fine tuning phase.
Finally the analysis of extracting reachable path is performed for autonomous navigation. It is observed that the edge preservation accuracy is high also when the model is exposed to new data which were not present while training the accuracy rate is maintained to be high.
This paper presents the Siamese Deep Convolutional Neural Network and its application in the indoor autonomous navigation for robots. This researches attempts to remove the drawback present in the previous system such as hardware and computational complexity, it also includes addressing of inhomogeneous texture of indoor scene. First an end to end trainable Siamese DCNN is preformed which can tackle semantic scene mapping and self-localization of robot so that the accuracy of indoor navigation is increased. In this research we focus on high degree of autonomy, which is a must for social robots. For training purpose autonomous exploration and unknown environments is used along with proper algorithm so that robot can adapt to unknown environments. For testing purpose, simulation is carried with sensor fusion method, so that real world noise can be reduced and accuracy can be increased. This dissertation focuses on the intelligent robot control in autonomous navigation tasks and investigates the robot learning in following aspects. This method is based on human instinct of imitation. In this standard real time data set is provided to the robot for training purpose, it gets train from these data and generalize over all unseen potential situations and environments. Convolutional Neural Network is used to determine the probability and based on that robot can act. After acceptable number of demonstrations, robot can predict output with high accuracy and hence can acquire the independent navigation skills. State-of-the-art reinforcement learning techniques is used to train the robot via interaction with the robots. Convolutional Neural Network is also incorporated for fast generalization. This model is designed in such a way that it can share maximum number of convolution layers among shared convolutional blocks, because of that proposed model is effect in terms of inference speed and memory. Also multi stage transfer leaning approach is adopted which efficiently generalized and proved promising results in experimental trial and simulation. Impact of transfer learning approach is highlighted in the paper and it is demonstrated by combining the semantic scene with scene classification is that high accuracy and robust navigation performance is achieved, without using additional offline camera calibration and sensors.