Drugtionary: Drug Pill Image Detection and Recognition based on Deep Learning

Abstract

Drugtionary, which is a mobile application, is developed to support people who lack medical understanding and avoid taking the wrong drug. It consists of four main features including (i) sign-up, (ii) managing profile and medication history, (iii) viewing medication information, and (iv) managing the schedule. For viewing medication information, there are three ways to retrieve the drug information--(i) text search, (ii) chatbot, and image search. We use string search and DialogFlow for text search and chatbot, respectively, whereas deep learning technique for image detection and recognition is used to search the given drug pill image. The experimental result shows that the model generated from the CenterNet method is suitable when compared to the Faster-RCNN, RetinaNet, Yolo, and SSD on our drug pill dataset. Moreover, our application is constructed by using React and React Native technology. All data are stored in the MongoDB database.