New-Generation Smart Visual Image Recognition Equipment Helps Differentiate Chinese Medicine Materials

IoT technology has brought many value-added features in modern life, one of which is the innovation to traditional culture. Indeed, the use of IoT has injected new insights into things people are already familiar with in their daily work. Traditional Chinese medicine is an industry that has existed for centuries. Though ancient, the industry also benefited from the development of IoT. In this regard, Dr. Chen, owner of Shengyuanzhen Chinese Medicine Store and assistant professor at the Dept. of Pharmacy, Jia Nan University of Pharmacy and Science, has taken advantage of his talent across pharmacy and technology to create a new path in traditional Chinese medicine and designed a "smart Chinese medicine visual image recognition equipment."

Dr. Chen's career is closely related to traditional Chinese medicine. Not only does he purchase medicinal materials for his store, but he also has in-depth contact with Chinese medicine in his daily career of teaching and research. However, traditional Chinese medicine identification suffers from substandard speed and accuracy, as many Chinese medicinal materials have similar appearances and are thus easily confused and difficult to distinguish from appearance. To solve this problem, the smart recognition equipment takes photos of Chinese medicines and uses image recognition algorithms to identify different types of Chinese medicines, helping professionals to correctly select and purchase the materials they need and reducing the waste of time and other resources.

Dr. Chen, who has previously conducted research on IoT technology and medical data, quickly started writing the first version of the code in Python. By integrating AI and IoT technologies, he sought to achieve real-time identification and monitoring of Chinese medicines, thereby improving the efficiency of QC with an innovative solution. The team has rapidly expanded to a complete lineup which includes software engineers, machine learning experts, electronic engineers, traditional Chinese medicine experts, and traditional Chinese medicine practitioners. The team members all work closely with each other and are each responsible for their own domain of development work. From basic hardware design to software development and application, they all endeavor to ensure the smooth operation of the system.

In the process of developing the equipment, the key technology is to use visual recognition functions with algorithms to correctly identify medicinal materials with similar appearances. Therefore, the team chose HUB 8735, a solution with strong visual recognition capabilities, in the development phase. HUB 8735 features built-in NPU AI computing engine, and can therefore accelerate the processing of AI models. Currently, it is widely used in various IoT scenarios which involve image recognition and AI computing.

During the interview, Dr. Chen also mentions the valuable experience he gained in the process of developing the smart recognition equipment. "When shooting the training materials for Chinese medicine recognition, we used high-performance photography equipment to capture fine images; however, as the smart recognition equipment has a relatively low-performance lens, there was a considerable gap between the captured images and the training images, resulting in poor recognition results," says Dr. Chen. In response, Dr. Chen suggests that when shooting training materials for image recognition, a device with image quality similar to the recognition lens should be chosen to achieve better recognition results.

During the design and development process of the smart Chinese medicine recognition equipment, the team found the solution of HUB 8735 through the III's IoT Service Hub. Later, a series of improvements were made with the advice of the expert consultation group, which helped turn the original prototype into a mass-production-ready model. After more than a year of optimization, LED lights and many other features were added, including panel integration, power supply and boost module improvements, connection to domestic IC EVB HUB8735, exterior design, etc. As of now, the product is almost ready for launching, and is now processing patent application.

As Dr. Chen talks about his goals for the next stage, in addition to continuously expanding the Chinese medicine training image database, he also wishes to further increase the types and patterns possible for recognition, so that he can keep updating the software, hardware, and database and optimize UX. Currently, the team is also considering the option to "do without local hardware" so that smartphones and other terminal devices can also run the visual recognition function of Chinese medicine. The main technical challenge now lies in how to overcome the problem of identification errors caused by color casts of the materials when photographed under different lighting conditions.

In addition to the smart Chinese medicine recognition equipment, Dr. Chen also shares some other applications of IoT technology in traditional Chinese medicine during the interview. For example, some medical institutions' Chinese medicine departments in Taiwan have introduced smart traditional Chinese medicine diagnosis and treatment systems. What's more, various companies have continued to develop innovative equipment for the four diagnostic methods of traditional Chinese medicine, i.e., observation, auscultation/olfaction, interrogation, and pulse feeling. In the end, Dr. Chen also expresses special thanks to Prof. Tung-Ti Chang (Chair of the School of Post-Baccalaureate Chinese Medicine, China Medical University), Prof. Chia-Hung Kao (Director of the AI Center, China Medical University Hospital), Prof. Tse-Yen Kenny Yang (Associate Professor at the Dept. of Medical Laboratory Science and Biotechnology, Asia University), and Prof. Tzu-Chueh Wang (Dean of the College of Pharmacy and Science, Jia Nan University of Pharmacy and Science) for inspiring him in his professional career.