Skip to main content

Multimodal Mining for Healthcare Group


The Multimodal Mining group at IBM Research - Almaden focuses on advanced analysis of multi-dimensional modalities including images (2D, 3D, 4D), videos, time series, and textual data. Members of the group are experienced researchers in the areas of medical imaging, computer vision, multimedia databases and signal processing with active contributions in these fields that can be found under our publications.

In the last 3-4 years, we have been exploring a novel use of content-based search techniques in clinical decision support in a project called AALIM (Advanced Analytics for Information Management, it means the knowledgeable one in many Asian languages). The key hypothesis we are exploring is whether disease-specific similarity in raw modality data can reveal similarity in patient diagnosis, and hence treatments and outcomes. Our approach leverages the consensus opinions from other physicians who looked at similar patients to allow a clinician to get a refined insight into the patient's diagnosis and the comparative effectiveness of different treatments and outcomes.

Much of the work in AALIM has revolved around analyzing different modality data to extract disease-specific information and to develop spatio-temporal descriptors for comparing modality data. Our current emphasis is on cardiology data with analysis of echocardiogram videos, electrocardiogram (EKG) time series, heart sounds, doppler imaging, cardiac MRI, etc. A unique aspect of our work is the end-to-end addressing of the decision support problem not only using novel techniques for disease-specific similarity search but also in the fusion of information from multiple cardiac modalities.

Please view our research for more information relating to our work.

[an error occurred while processing this directive]