This paper investigates sequential robotic microassembly for the construction of 3D micro-electro-mechanical systems (MEMSs) structures using a 3D visual servoing approach. The previous solutions proposed in the literature for these kinds of problems are based on 2D visual control because of the lack of precise and robust 3D measures from the work scene. In this paper, the relevance of the real-time 3D visual tracking method and the 3D vision-based control law proposed is demonstrated. The 3D poses of the MEMSs are supplied in real-time by a computer-aided design model-based tracking algorithm. This algorithm is sufficiently accurate and robust to enable a precise regulation toward zero of the 3D error using the proposed pose-based visual servoing approach. Experiments on a microrobotic setup have been carried out to achieve assemblies of two or more 400 μm × 400 μm × 100 μm silicon micro-objects by their respective 97 μm × 97 μm × 100 μm notches with an assembly clearance from 1 μm to 5 μm. The different microassembly processes are performed with a mean error of 0.3 μm in position and 0.35×10−2rad in orientation.
Abbott, J.J., Nagy, Z., Beyeler, F., and Nelson, B.J. ( 2007). Robotics in the small, Part I: Microrobotics. IEEE Robotics and Automation Magazine, 14(2): 92-103.
2.
Agnus, J., Nectoux, P., and Chaillet, N. ( 2005). Overview of microgrippers and design of a micromanipulation station based on mmoc microgripper. IEEE International Symposium on Computational Intelligence in Robotics and Automation, Finland, pp. 117-123.
3.
Ammi, M. and Ferreira, A. ( 2007). Robotic assisted micromanipulation system using virtual fixtures and metaphors. IEEE International Conference Robotics and Automation pp. 454-460 Rome, Italy.
4.
Ammi, M., Fremont, V., and Ferreira, A. ( 2005). Flexible microscope calibration using virtual pattern for 3D telemicromanipulation. IEEE International Conference Robotics and Automation, pp. 3888-3893 Barcelona, Spain.
5.
Chaumette, F. and Hutchinson, S. ( 2006). Visual servo control, Part 1: Basic approaches . IEEE Robotics and Automation Magazine, 13(4): 82-90.
6.
Chaumette, F. and Hutchinson, S. ( 2007). Visual servo control, Part II: Advanced approaches . IEEE Robotics and Automation Magazine, 14(1): 109-118.
7.
Chen, L., Sun L., Weibin, R., and Xinqian, B., (2004). Hybrid control of vision and force for MEMS assembly system. IEEE International Conference on Robotics and Biometrics, Shenyang, China, pp. 136-141.
8.
Cohn, M.B., Böhringer, K.F., Noworolski, J.M., Singh, A., and Keller, C.G. ( 1998). Microassembly technologies for MEMS. SPIE Microelectronic Structures and MEMS for Optical Processing IV, Santa Clara, CA, Vol. 3513, pp. 6-12.
9.
Comport, A., Marchand, E., Pressigout, M., and Chaumette, F. ( 2006). Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Transactions on Visualization and Computer Graphics, 12(4): 615-628.
10.
del Corrala, C., Estebana, P.J., Albuta, A., Zhoua, Q., and Koivob, H.N. ( 2002). Influence of environment conditions on microassembly components and pick-and-place operations. Proceedings of the Fourth International Conference on Machine Automation, pp. 267-274.
11.
Devanathan, R., Wenting, S., Chai, S., and Shacklock, A. ( 2006). Multi view and multi scale image based visual servo for micromanipulation. Studies in Computational Intelligence , 8: 105-133.
12.
Drummond, T. and Cipolla, R. ( 2002). Real-time visual tracking of complex structures . IEEE Transactions on Pattern Analysis and Machine Intelligence , 24(7): 932-946.
13.
Fatikow, S., Buerkle, A., and Seyfried, J. (( 1999)). Automatic control system of a microrobot-based microassembly station using computer vision. SPIE conference on Microrobotics and Microassembly, Boston, MA, Vol. 3834, pp. 11-22.
14.
Fatikow, S., Wortmann, T., Mikczinski, M., Dahmen, C., and Stolle, C. ( 2009). Towards automated robot based nanohandling. Proceedings of IEEE Chinese Control and Decision Conference (CCDC) , China, pp. 5039-5044.
15.
Feddema, J.T. and Simon, R.W. ( 1998). Visual servoing and CAD-driven microassembly. IEEE Robotics and Automation Magazine, 5(4): 18-24.
16.
Figl, M., Ede, C., Hummel, J., Wanschitz, F., Ewers, R., Bergmann, H., and Birkfellner, W. ( 2005). A fully automated calibration method for an optical see-through head-mounted operating microscope with variable zoom and focus . IEEE Transactions on Medical Imaging, 24(11): 1492-1499.
17.
Greminger, M.A. and Nelson, B.J. ( 2004). Vision-based force measurement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(3): 290-298.
18.
Greminger, M.A. and Nelson, B.J. ( 2005). A deformable object tracking algorithm robust to occlusions and spurious edges. IEEE International Conference on Robotics and Automation, Barcelona, Spain, pp. 1264-1269.
19.
Haralick, R., Joo, H., Lee, C., Zhuang, X., Vaidya, V., and Kim, M. ( 1989). Pose estimation from corresponding point data. IEEE Transactions on Systems, Man and Cybernetics, 19(6): 1426-1445.
20.
Huber, P.-J. ( 1981). Robust Statistics. Wiley, New York.
21.
Hutchinson, S., Hager, G., and Corke, P. ( 1996). A tutorial on visual servo control. IEEE Transactions on Robotics and Automation, 12(5): 651-670.
22.
Jasper, D. and Fatikow, S. ( 2009). High-speed position tracking for nanohandling inside scanning electron microscopes. Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Japan, pp. 508-513.
23.
Kawaji, F. and Fukuda, T. ( 2001). 3D attitude control system for biomicromanipulation . Proceeding of International Symposium of Micromechatronics and Human Science , Nagoya, Japan, pp. 197-202.
24.
Kratochvil, B.E., Dong, L.X., and Nelson, B.J. ( 2009). Real-time rigid-body visual tracking in a scanning electron microscope. International Journal of Robotics Research , 28(4): 498-511.
25.
Lowe, D.G. ( 1991). Fitting parameterized three-dimensional models to images . IEEE Transactions on Pattern Analysis and Machine Intelligence , 13(5): 441-450.
26.
Malis, E., Chaumette, F., and Boudet, S. ( 1999). 2 1/2 D visual servoing. IEEE Transactions on Robotics and Automation, 15(2): 238-250.
27.
Marchand, E., Chaumette, F., Spindler, F., and Perrier, M. ( 2002). Controlling an uninstrumented manipulator by visual servoing . International Journal of Robotics Research, 21(7): 635-648.
28.
Martinet, P. and Gallice, J. ( 1999). Position based visual servoing using a non-linear approach . IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, South Korea, pp. 531-536.
29.
Murray, R., Zexiang, L., and Sastry, S. ( 1994). A Mathematical Introduction to Robotic Manipulation . CRC PressBoca Raton, FL, USA.
30.
Nelson, B.J., Ralis, S., Zhou, Y., and Vikramaditya, B. (1999). Force and vision feedback for robotic manipulation of the microworld. Experimental Robotics VI - Lectures Notes in Control and Information Sciences, 250: 433-442.
31.
Popa, D.O. and Stephanou, H.E. ( 2004). Micro and mesoscale robotic assembly. Journal of Manufacturing Process, 6(1): 52-71.
32.
Probst, M., Hürzeler, C., Borer, R., and Nelson, B.J. ( 2009). A microassembly system for the flexible assembly of hybrid robotic MEMS devices. International Journal of Optomechatronics , 3(2): 69-90.
33.
Ralis, S., Vikramaditya, B., and Nelson, B.J. ( 2000). Micropositioning of a weakly calibrated microassembly system using coarse-to-fine visual servoing strategies. IEEE Transactions on Electronics Packaging Manufacturing, 23(2): 123-131.
34.
Stoeter, S.A., Probst, M., Nelson, B.J., and Iranzo, M.A. ( 2006). Improving tracking precision for microassembly . Proceedings of the 2006 IEEE International Conference on Robotics and Automation, Orlando, FL.
35.
Sun, W. and Chin, T. ( 2004). Image-based visual servo for micromanipulation: a multiple-view and multiple-scale approach. International Symposium on Micro-Nanomechatronics and Human Science, pp. 341-346.
36.
Tamadazte, B., Dembélé, S., and Fort-Piat, N.L. ( 2008a). A multiscale calibration of a photon video microscope for visual servo control: Application to micromanipulation. IEEE International Workshop on Robotic and Sensors Environments , pp. 29-34 Ottawa, Canada.
37.
Tamadazte, B., Dembélé, S., Fortier, G., and Le Fort-Piat, N. (2008b). Automatic micromanipulation using multiscale visual servoing. IEEE Conference on Automation Science and Engineering, Washington, USA, pp. 977-982.
38.
Tamadazte, B., Dembélé, S., and Fort-Piat, N.L. ( 2009). A multiscale calibration of a photon videomicroscope for visual servo control: Application to mems micromanipulation and microassembly . Sensors and Transducers Journal, 5: 37-52.
39.
Tao, X., Cho, H., and Cho, Y. ( 2005). Microassembly of peg and hole using active zooming . Proceedings of SPIE, 6052, 605204.1-605204.12Sapporo, Japan.
40.
Wechsung, R. ( 2002). Market analysis for microsystems 2002- 2005. A Report from the NEXUS Task Force. MST.
41.
Wilson, W., Hulls, C., and Bell, G. ( 1996). Relative end-effector control using cartesian position-based visual servoing. IEEE Transaction on Robotics and Automation , 12: 684-696.
42.
Yang, G., Gaines, J., and Nelson, B.J. ( 2003). A surpervisory wafer-level 3D microassembly system for hybrid MEMS fabrication. Journal of Intelligent and Robotic Systems, 37: 43-68.
43.
Yang, G., Gaines, J., and Nelson, J.B. ( 2005). Optomechatronic design of microassembly systems for manufacturing hybrid microsystems. IEEE Transactions on Industrial Electronics , 52: 1013-1023.
44.
Yesin, K. and Nelson, B. ( 2005). A CAD-model based tracking system for visually guided microassembly. Robotica, 23: 409-418.
45.
Yesin, K.B. and Nelson, B.J. ( 2004). Robust CAD model-based visual tracking for 3D microassembly using image space potentials. IEEE International Conference on Robotics and Automation, New Orleans, USA, pp. 1868-1873.
46.
Zhou, Q. and Chang, B. ( 2006). Microhandling using robotic manipulation and capillary self-alignment. EEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, pp. 5883-5888.
47.
Zhou, Q., Corral, C., Esteban, P., Albut, A., and Koivo, H. ( 2002). Environmental influences on microassembly. IEEE/RSJ International Conference on Intelligent Robots and System , pp. 1760-1765.
48.
Zhou, Y. and Nelson, B.J. ( 1999). Calibration of a parametric model of an optical microscope . Optical Engeenering, 38(12): 1989-1995.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
