Full metadata record
DC Field | Value | Language |
---|---|---|
dc.contributor.author | 이연준 | - |
dc.date.accessioned | 2023-12-21T07:32:14Z | - |
dc.date.available | 2023-12-21T07:32:14Z | - |
dc.date.issued | 2023-10 | - |
dc.identifier.citation | IEEE Access, v. 11, Page. 116721.0-116731.0 | - |
dc.identifier.issn | 2169-3536 | - |
dc.identifier.uri | https://ieeexplore.ieee.org/document/10286540 | en_US |
dc.identifier.uri | https://repository.hanyang.ac.kr/handle/20.500.11754/187667 | - |
dc.description.abstract | Chronic otitis media is characterized by recurrent infections, leading to serious complications, such as meningitis, facial palsy, and skull base osteomyelitis. Therefore, active treatment based on early diagnosis is essential. This study developed a multi-modal multi-fusion (MMMF) model that automatically diagnoses ear diseases by applying endoscopic images of the tympanic membrane (TM) and pure-tone audiometry (PTA) data to a deep learning model. The primary aim of the proposed MMMF model is adding "normal with hearing loss" as a category, and improving the diagnostic accuracy of the conventional four ear diseases: normal, TM perforation, retraction, and cholesteatoma. To this end, the MMMF model was trained on 1,480 endoscopic images of the TM and PTA data to distinguish five ear disease states: normal, TM perforation, retraction, cholesteatoma, and normal (hearing loss). It employs a feature fusion strategy of cross-attention, concatenation, and gated multi-modal units in a multi-modal architecture encompassing a convolutional neural network (CNN) and multi-layer perceptron. We expanded the classification capability to include an additional category, normal (hearing loss), thereby enhancing the diagnostic performance of extant ear disease classification. The MMMF model demonstrated superior performance when implemented with EfficientNet-B7, achieving 92.9% accuracy and 90.9% recall, thereby outpacing the existing feature fusion methods. In addition, five-fold cross-validation experiments were conducted, in which the model consistently demonstrated robust performance when endoscopic images of the TM and PTA data were applied to the deep learning model across all datasets. The proposed MMMF model is the first to include a category of normal ear disease state with hearing loss. The developed model demonstrated superior performance compared to existing CNN models and feature fusion methods. Consequently, this study substantiates the utility of simultaneously applying PTA data and endoscopic images of the TM for the automated diagnosis of ear diseases in clinical settings and validates the usefulness of the multi-fusion method. Author | - |
dc.description.sponsorship | This work was supported in part by the Institute of Information and Communications Technology Planning and Evaluation (IITP) funded by the Korean Government for the Ministry of Science and ICT (MSIT), South Korea, through the Artificial Intelligence Convergence Innovation Human Resources Development funded by Hanyang University (ERICA) under Grant RS-2022-00155885; in part by the Artificial Intelligence Convergence Research Center, Hanyang University (ERICA), under Grant 2020-0-01343; in part by the National Research Foundation of Korea (NRF) funded by the Korean Government (MSIT) under Grant NRF-2022R1F1A1074999; in part by the Korea University Grant and the Medical Data-Driven Hospital Support Project through the Korea Health Information Service (KHIS)funded by the Ministry of Health and Welfare, Republic of Korea; and in part by MSIT under the ICT Challenge and Advanced Network(ICAN) of HRD Program Supervised by IITP under Grant IITP-2022-RS-2022-00156439. | - |
dc.language | en | - |
dc.publisher | Institute of Electrical and Electronics Engineers Inc. | - |
dc.subject | Deep learning | - |
dc.subject | Artificial intelligence | - |
dc.subject | Auditory system | - |
dc.subject | Biomedical imaging | - |
dc.subject | Bones | - |
dc.subject | Classification algorithms | - |
dc.subject | Computer aided diagnosis | - |
dc.subject | Convolutional neural networks | - |
dc.subject | Data models | - |
dc.subject | Diseases | - |
dc.subject | Ear | - |
dc.subject | Electronic medical records | - |
dc.subject | Media | - |
dc.title | Toward better ear disease diagnosis: A multi-modal multi-fusion model using endoscopic images of the tympanic membrane and pure-tone audiometry | - |
dc.type | Article | - |
dc.relation.volume | 11 | - |
dc.identifier.doi | 10.1109/ACCESS.2023.3325346 | - |
dc.relation.page | 116721.0-116731.0 | - |
dc.relation.journal | IEEE Access | - |
dc.contributor.googleauthor | Kim, Taewan | - |
dc.contributor.googleauthor | Kim, Sangyeop | - |
dc.contributor.googleauthor | Kim, Jaeyoung | - |
dc.contributor.googleauthor | Lee, Yeonjoon | - |
dc.contributor.googleauthor | Choi, June | - |
dc.sector.campus | E | - |
dc.sector.daehak | 소프트웨어융합대학 | - |
dc.sector.department | 컴퓨터학부 | - |
dc.identifier.pid | yeonjoonlee | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.