Period
2022.10-2023.10
Cooperative unit
The First Affiliated Hospital of USTC
Introduction
Neonatal jaundice is the most common clinical symptom in the neonatal period, and its cause is hyperbilirubinemia, which can lead to bilirubin encephalopathy, resulting in serious neurological sequelae that can follow the child for life. The disease has a high incidence, rapid development and serious consequences, and poses a great danger to the life and health of newborns, which is highly valued by the national authorities, who have put forward the strategic requirement of making China free of kernicterus. At present, the monitoring of nuclear jaundice relies excessively on hospital equipment, and there is a shortage of resources and geographical limitations. This project proposes to carry out research on a convenient home monitoring system for nuclear jaundice based on multimodal data fusion. The project focuses on building a regression model based on skin color-color card and a multimodal data fusion model based on attention mechanism to realize the organic fusion of multimodal data such as skin photos and physiological status of newborns; it focuses on studying the quantitative assessment mechanism for mobile terminal, quantifying the regression results, generating disease research and judgment reports, and developing a supporting cell phone APP to meet the demand for real-time monitoring at home. The research results can realize portable, inexpensive, easy-to-use, relatively accurate kernicterus monitoring and smartphone-based optical bilirubin assessment, which is important for reducing the incidence of kernicterus in China.
Methodology
This project will conduct research on several aspects of practical scenarios, key technologies and software development applications for home monitoring of neonatal jaundice. The main research contents include:
1. Estimation of bilirubin levels based on skin tone - colorimetric card images: (1)Combining a large amount of unlabeled skin color-colorimetric card image data, a generalized base representation is trained using a self-supervised technique. (2) The Transformer-based neural network model was further trained on the generic base representation using the acquired skin color-colorimetric card images and the corresponding bilirubin gold standard determination dataset.
2. Construction of a high-precision bilirubin level regression model based on multimodal fusion for mobile: (1)Using the skin color-colorimetric card image as the primary information, mapping each physiological state text data into high-dimensional embedded information, modulating the primary information with text information through attention mechanism, and realizing the comprehensive consideration of multidimensional and multimodal information. (2) Using the trained high-precision bilirubin regression model as the teacher model, we design the student model that can be deployed and run in real time on the mobile terminal, and realize the knowledge migration from the teacher model to the student model through knowledge distillation.
3. Develop a mobile-oriented jaundice classification assessment application: Using the mature image acquisition quality determination technology, we can determine and guide the quality of skin color-color card images taken by users. Provide intelligent prompts to guide users to fill in relevant physiological status information, provide users with jaundice risk estimation and provide reference suggestions based on criteria.
