Iranian Journal of Mechanical Engineering Transactions of the ISME

Iranian Journal of Mechanical Engineering Transactions of the ISME

Integrating Adaptive Sliding Mode Control and Deep Learning for Autonomous Vision-driven Fruit Sorting robot

Document Type : Research Paper

Authors
Hassan Sayyaadi 1
Sara Adeli 2
1 Professor, Department of Mechanical Engineering, Sharif University of Technology, Tehran, Iran
2 M.Sc., Student, Department of Mechanical Engineering, Sharif University of Technology, Tehran, Iran
10.30506/jmee.2025.2037549.1353
Abstract
This article presents a novel application of deep learning in automated fruit-sorting robotics, improving real-time object recognition and handling. By integrating advanced neural networks, the robot achieves higher accuracy in identifying various fruits, addressing fruit variability and enhancing sorting precision. This innovation, combining deep learning and visual servoing, represents a significant advancement in automated fruit-sorting technology, with promising benefits for agricultural processes. The project aims to control a fruit-sorting robot using image processing data, merging sliding mode control and NN-based (neural network-based) techniques for automation. Utilizing the latest YOLO (You Only Look Once) model, the system classifies and positions fruits rapidly and accurately, making it suitable for real-time applications. After identifying fruit types and positions, a controller is designed for the pick-and-place process. Sliding mode control manages uncertainties and guides manipulator movements precisely, while a neural network controls joint angles for smooth and accurate fruit manipulation. Comparative tests on a simulated robot revealed that the NN-based controller excels in accuracy and speed, adapting to different fruit configurations effectively. The sliding mode controller, though robust and stable, is sensitive to uncertainties, affecting sorting precision. The hybrid system, integrating both controllers, enhances adaptability by combining the NN-based approach's precision with the stability of sliding mode control, optimizing fruit-sorting performance across diverse scenarios. Results emphasize selecting the appropriate controller to balance precision and speed based on specific application needs.
Keywords
Fruit detection
NN-based control
Adaptive sliding mode control
Visual servoing

Subjects

[1] V. E. Balas, R. Kumar, and R. Srivastava, "Recent Trends and Advances in Artificial Intelligence and Internet of Things", in Intelligent Systems Reference Library (ISRL), Vol. 172, Cham: Springer International Publishing, 2020, doi: 10.1007/978-3-030-32644-9.
 
[2] B. Iscimen, H. Atasoy, Y. Kutlu, S. Yildirim, and E. Yildirim, “Smart Robot Arm Motion using Computer Vision,” Elektronika ir Elektrotechnika, Vol. 21, No. 6, pp. 3-7, 2015, doi: 10.5755/j01.eee.21.6.13749.
 
[3] N. Kondo, “Robotization in Fruit Grading System,” Sensing and Instrumentation for Food Ruality and Safety, Vol. 3, No. 1, pp. 81-87, 2009, doi: 10.1007/s11694-008-9065-x.
 
[4] M. M. Sofu, O. Er, M. C. Kayacan, and B. Cetişli, “Design of an Automatic Apple Sorting System using Machine Vision,” Computers and Electronics in Agriculture, Vol. 127, pp. 395-405, 2016, doi: 10.1016/j.compag.2016.06.030.
 
[5] T. Dewi, P. Risma, and Y. Oktarina, “Fruit Sorting Robot Based on Color and Size for an Agricultural Product Packaging System,” Bulletin of Electrical Engineering and Informatics, Vol. 9, No. 4, pp. 1438-1445, 2020, doi: 10.11591/eei.v9i4.2353.
 
[6] V. D. Cong, “Visual Servoing Control of 4-DOF Palletizing Robotic arm for Vision Based Sorting Robot System,” International Journal on Interactive Design and Manufacturing (IJIDeM), Vol. 17, No. 2, pp. 717-728, Apr. 2023, doi: 10.1007/s12008-022-01077-8.
 
[7] W. T. Abbood, O. I. Abdullah, and E. A. Khalid, “A Real-time Automated Sorting of Robotic Vision System Based on the Interactive Design Approach,” International Journal on Interactive Design and Manufacturing (IJIDeM), Vol. 14, No. 1, pp. 201-209, 2020, doi: 10.1007/s12008-019-00628-w.
 
[8] H. M. T. Abbas, U. Shakoor, M. J. Khan, M. Ahmed, and K. Khurshid, “Automated Sorting and Grading of Agricultural Products Based on Image Processing,” 8th International Conference on Information and Communication Technologies (ICICT), pp. 78-81, 16-17 November 2019, Karachi, Pakistan, doi: 10.1109/ICICT47744.2019.9001971.
 
[9] P. Chen, and V. Elangovan, “Object Sorting using Faster R-CNN,” International Journal of Artificial Intelligence & Applications (IJAIA), Vol. 11, No. 5/6, pp. 27-36, 2020, doi: 10.5121/ijaia.2020.11603.
 
[10] S. Sanwar, and M. I. Ahmed, “Automated Object Sorting System with Real-time Image Processing and Robotic Gripper Mechanism Control,” Journal of Engineering Advancements, Vol. 04, No. 03, pp. 70-79, 2023, doi: 10.38032/jea.2023.03.003.
 
[11] A. Visioli, and G. Legnani, “On the Trajectory Tracking Control of Industrial SCARA Robot Manipulators,” IEEE Transactions on Industrial Electronics, Vol. 49, No. 1. pp. 224-232, 2002, doi: 10.1109/41.982266.
 
[12] P. Bobka, F. Gabriel, and K. Dröder, “Fast and Precise Pick and Place Stacking of Limp Fuel Cell Components Supported by Artificial Neural Networks,” CIRP Annals, Vol. 69, No. 1, pp. 1-4, 2020, doi: 10.1016/j.cirp.2020.04.103.
 
[13] M. E. Mostafa, A. E. Mostafa, H. H. Ammar, and R. Shalaby, "ANN-Based Modeling and Control of a Pick and Place Manipulato", In: X. S. Yang, R. S. Sherratt, N. Dey, A. Joshi, (eds) Proceedings of Eighth International Congress on Information and Communication Technology, ICICT, Lecture Notes in Networks and Systems, Springer, Singapore, Vol. 694, pp. 189-204, 2023, https://doi.org/10.1007/978-981-99-3091-3_15.
 
[14] S. Patil, S. Waghule, S, Waje, P, Pawar, and S, Domb, “Efficient Object Detection with YOLO: A Comprehensive Guide,” International Journal of Advanced Research in Science, Communication and Technology, pp. 519-531, 2024, doi: 10.48175/IJARSCT-18483.
 
[15] F. Piltan, S. Emamzadeh, Z. Hivand, F. Shahriyari, and M. Mirazaei “PUMA-560 Robot Manipulator Position Sliding Mode Control Methods using MATLAB / SIMULINK and Their Integration into Graduate / Undergraduate Nonlinear Control, Robotics and MATLAB Courses,” International Journal of Robotic and Automation, (IJRA),” Vol. 6, No. 3, pp. 106-150, 2012.
 
[16] B. Armstrong, O. Khatib, and J. Burdick, “The Explicit Dynamic Model and Inertial Parameters of the PUMA 560 Arm,” in Proceedings, 1986 IEEE International Conference on Robotics and Automation, Institute of Electrical and Electronics Engineers, pp. 510-518, San Francisco, CA, USA, 1986, doi: 10.1109/ROBOT.1986.1087644.