Design and Implementation of a Control Device for Intelligent Wheelchair by Combining Image Processing and Acceleration Sensor

Document Type : Research Paper


1 M.Sc. Student, Department of Mechanical Engineering, Pardis Branch, Islamic Azad University, Tehran, Iran

2 Corresponding Author, Assistant Professor, Department of Mechanical Engineering, Pardis Branch, Islamic Azad University, Tehran, Iran


In this paper, a gadget for controlling an electric wheelchair (EWC) is designed.
The device is designed based on the combination of acceleration sensor data from
head rotation, and image processing data from user’s face recognition, for
commanding to the EWC. This gadget designed as a wearable device and is
developed low cost, safe, and flexible for the patients with spinal cord injuries as
well as the elderly with limited hand use. The mechanical design, sensor tuning,
and 3Dprinted prototype of the gadget are presented. Finally the result of
experimental test is discussed. Acceleration sensor module and camera and
Raspberry Pi board are the core of the gadget. For performance evaluation,
several experiments have been performed. The commands sent by the user are
divided into four control commands (right, stop, move, left). Forward movement
command, is performed by showing a happy expression on the face. The
experiments show that the head angle controller and the image processor react
to the stop in the shortest time, which indicates that it has a high level of safety.


Main Subjects

[1] Ghorbel, A., Amor, N.B., and Jallouli, M., "A Survey on Different Human-machine
Interactions used for Controlling an Electric Wheelchair", Procedia Computer Science, Vol.
159, pp. 398-407, (2019).
[2] Dahmani, M, Chowdhury, M.E., Khandakar. A., Rahman, T., Al-Jayyousi, K., Hefny, A.,
and Kiranyaz, S., "An Intelligent and Low-cost Eye-tracking System for Motorized Wheelchair
Control", Sensors, Vol. 20, pp. 3936, (2020).
[3] Tiwari, P.K., Choudhary, A., Gupta, S., Dhar, J., and Chanak, P., "Sensitive Brain-
Computer Interface to Help Maneuver a Miniature Wheelchair using Electroencephalography",
2020 IEEE International Students' Conference on Electrical, Electronics and Computer Science
(SCEECS), pp. 1-6, (2020).
[4] Al-Qaysi, Z.T., Zaidan, B.B., Zaidan, A.A., and Suzani, M.S., "A Review of Disability
EEG Based Wheelchair Control System: Coherent Taxonomy, Open Challenges and
Recommendations", Computer Methods and Programs in Biomedicine, Vol. 164, pp. 221-237,
[5] Tang, J., Liu, Y., Hu, D., and Zhou, Z., "Towards BCI-actuated Smart Wheelchair System",
Biomedical Engineering Online, Vol. 17, pp. 1-22, (2018).
[6] Cruz, R., Souza, V., Bastos Filho, T., and Lucena, V., "Electric Powered Wheelchair
Command by Information Fusion from Eye Tracking and BCI", 2019 IEEE International
Conference on Consumer Electronics (ICCE), January 11-13, 2019, Las Vegas, USA, pp. 1-2,
[7] Wasim, M., Khan, J., Saeed, D., and Usman, D., "Eye Controlled Mobile Robot with
Shared Control for Physically Impaired People", International Journal of Advanced Computer
Science and Applications, Vol. 8, (2017).
[8] Singh, A., Choudhary, P., Shilpi Basuri, C.U., Basuri, S., and Umadi, C., "Review on Brain
Controlled Wheelchair", IJRAR-International Journal of Research and Analytical Reviews
(IJRAR), Vol. 7, pp. 585-589, (2020).
[9] Kounte, M.R., Tripathy, P.K., Pramod, P., and Bajpai, H., "Implementation of Brain
Machine Interface using Mind Wave Sensor", Procedia Computer Science, Vol. 171, pp. 244-
252, (2020).
[10] Awais, M.A., Yusoff, M.Z., Yahya, N., Ahmed, S.Z., and Qamar, M.U., "Brain Controlled
Wheelchair: A Smart Prototype", Journal of Physics: Conference Series, Vol. 1529, pp. 042075,
[11] Voznenko, T.I., Chepin, E.V., and Urvanov, G.A., "The Control System Based on
Extended BCI for a Robotic Wheelchair", Procedia Computer Science, Vol. 123. pp. 522-527,
[12] AlAbboudi, M., Majed, M., Hassan, F., and Nassif, A.B., "EEG Wheelchair for People of
Determination", 2020 Advances in Science and Engineering Technology International
Conferences (ASET), March 18-19, Dubai, United Arab Emirates, pp. 1-5, (2020).
[13] Rakasena, E.P.G., and Herdiman, L., "Electric Wheelchair with Forward-reverse Control
using Electromyography (EMG) Control of Arm Muscle", Journal of Physics: Conference
Series Vol. 1450, pp. 012118, (2020).
[14] Cheraghpour, F., Farzad, F., Shahbabai, M., and Alashti, M. R. S., "FARAT1: an Upper
Body Exoskeleton Robot", 2017 5th RSI International Conference on Robotics and
Mechatronics (ICRoM), August 10-12, Tehran, Iran, pp. 463-468, (2017).
[15] Ben Khelifa, M.M., Lamti, H.A., and Hugel, V., "A Muscular and Cerebral Physiological
Indices Assessment for Stress Measuring During Virtual Wheelchair Guidance", Brain
Sciences, Vol. 11, pp. 274, (2021).
[16] Morón, J., DiProva, T., Cochrane, J.R., Ahn, I.S., and Lu, Y., "EMG-based Hand Gesture
Control System for Robotics", 2018 IEEE 61st International Midwest Symposium on Circuits
and Systems (MWSCAS), August 9-12, Lansing, MI, USA, pp. 664-667, (2018).
[17] Lv, Z., Zhang, C., Zhou, B., Gao, X., and Wu, X., "Design and Implementation of an Eye
Gesture Perception System Based on Electrooculography", Expert Systems with Applications,
Vol. 91, pp. 310-321, (2018).
[18] Rajesh, A., and Mantur, M., "Eyeball Gesture Controlled Automatic Wheelchair using
Deep Learning", 2017 IEEE Region 10 Humanitarian Technology Conference (R10-HTC),
August 10-12, New York, USA, pp. 387-391, (2017).
[19] Rabhi, Y., Mrabet, M., Fnaiech, F., Gorce, P., Miri, I., and Dziri, C., "Intelligent
Touchscreen Joystick for Controlling Electric Wheelchair", Irbm, Vol. 39, pp. 180-193, (2018).
[20] Madona, P., Nisa, H.K., Wijaya, Y.P., and Akhyan, A., "The Design of Wheelchair
Systems with Raspberry Pi 3-Based Joystick Analog and Voice Control", IOP Conference
Series: Materials Science and Engineering, Vol. 846, pp. 012032, (2020).
[21] Anwer, S., Waris, A., Sultan, H., Butt, S.I., Zafar, M. H., Sarwar, M., and Pujari, A.N.,
"Eye and Voice-controlled Human Machine Interface System for Wheelchairs using Image
Gradient Approach", Sensors, Vol. 20, pp. 5510, (2020).
[22] Renuka, K., Harini, R., Balaji, V., and Ashok, N., "Raspberry Pi Based Multi-optional
Wireless Wheelchair Control and Gesture Recognized Home Assist System", IOP Conference
Series: Materials Science and Engineering, Vol. 1084, pp. 012071, (2021).
[23] Poornima, G., "Information Fusion Based Wheelchair Control for Paralyzed Patient",
2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)
(I-SMAC), October 10-13, Palladam, India, pp. 921-927, (2020).
[24] Al-Nabulsi, J.I., "A Novel Approach to Wheelchair Design and Operation using Multi-
Function Controller", Journal of Computer Science, Vol. 16, pp. 1029-1041, (2020).
[25] Abu zaher, M., and Al-Azzeh, J., "Mind-Wave Wheelchair System", International Review
on Modelling and Simulations, (IREMOS), Vol. 14(1), pp. 18-23, (2021).
[26] Zubair, Z.R.S., "A Deep Learning Based Optimization Model for Based Computer
Interface of Wheelchair Directional Control", Tikrit Journal of Pure Science, Vol. 26, pp. 108-
112, (2021).
[27] Haddad, M., Sanders, D., Ikwan, F., Thabet, M., Langner, M., and Gegov, A., "Intelligent
HMI and Control for Steering a Powered Wheelchair using a Raspberry Pi Microcomputer",
2020 IEEE 10th International Conference on Intelligent Systems (IS), October 10-13, Palladam,
India, pp. 223-228, (2020).
[28] Haddad, M., Sanders, D., Langner, M., Omoarebun, P., Thabet, M., and Gegov, A., "Initial
Results from using an Intelligent System to Analyse Powered Wheelchair Users’ Data", 2020
IEEE 10th International Conference on Intelligent Systems (IS), October 10-13, Palladam,
India, pp. 241-245, (2020).
[29] Rabhi, Y., Mrabet, M., and Fnaiech, F., "A Facial Expression Controlled Wheelchair for
People with Disabilities", Computer Methods and Programs in Biomedicine, Vol. 165, pp. 89-
105, (2018).
[30] Veerati, R., Suresh, E., Chakilam, A., and Ravula, S. P., "Eye Monitoring Based Motion
Controlled Wheelchair for Quadriplegics", Microelectronics, Electromagnetics and
Telecommunications, pp. 41-49, (2018).
[31] Juhong, A., Treebupachatsakul, T., and Pintavirooj, C., "Smart Eye-tracking System",
2018 International Workshop on Advanced Image Technology (IWAIT), January 7-9, Chiang
Mai, Thailand, pp. 1-4, (2018).
[32] Sharma, J., Anbarasu, M., Chakraborty, C., and Shanmugasundaram, M., "Iris Movement
Based Wheel Chair Control using Raspberry Pi", IOP Conference Series: Materials Science and
Engineering, Vol. 263, pp. 052049, (2017).
[33] Theivapriya, V., Jagatheeshwari, S., Aishwarya, D., and Sudha, G., "Smart Wheel Chair
for Quadriplegics", 2018 International Conference on Communication, Computing and Internet
of Things (IC3IoT), February 17-19, Chennai, India, pp. 455-458, (2018).
[34] Zhang, H., Wang, J., Liu, J., and Chen, D., "Design of a Low-cost Eye-movementcontrolled
Wheelchair Based on Near Infrared Light Sensor and Arduino", DEStech
Transactions on Computer Science and Engineering, (aiea), (2017).
[35] Sharifuddin, M.S.I., Nordin, S., and Ali, A.M., "Comparison of CNNs and SVM for Voice
Control Wheelchair", IAES International Journal of Artificial Intelligence, Vol. 9, pp. 387,
[36] Hasan, S., Faisal, F., Sabrin, S., Tong, Z., Hasan, M., Debnath, D., and Alam, J., "A
Simplified Approach to Develop Low Cost Semi-Automated Prototype of a Wheelchair",
University of Science and Technology Annual (USTA), (2020).
[37] Dey, P., Hasan, M.M., Mostofa, S., and Rana, A.I., "Smart Wheelchair Integrating Head
Gesture Navigation", 2019 International Conference on Robotics, Electrical and Signal
Processing Techniques (ICREST), January 7-9, Chiang Mai, Thailand, pp. 329-334, (2019).
[38] Machangpa, J.W., and Chingtham, T.S., "Head Gesture Controlled Wheelchair for
Quadriplegic Patients", Procedia Computer Science, Vol. 132, pp. 342-351, (2018).
[39] Mavuş, U., and Sezer, V., "Head Gesture Recognition via Dynamic Time Warping and
Threshold Optimization", March 27-31, Savannah, GA, USA, 2017 IEEE Conference on
Cognitive and Computational Aspects of Situation Management (CogSIMA), pp. 1-7, (2017).
[40] Viola, P., and Jones, M., "Robust Real-time Object Detection", International Journal of
Computer Vision, Vol. 4, pp. 4, (2001).