Multi-Modal Physiological Data Fusion and Multi-Agent Clinical Decision Support for Intelligent Patient Monitoring

Abstract

Patient monitoring is a cornerstone of clinical care, particularly in intensive care units, post-operative recovery wards, and settings managing patients with chronic diseases. Traditional bedside monitoring systems rely on single-modality physiological signals—such as electrocardiogram (ECG) for cardiac rhythm or pulse oximetry for oxygen saturation—and generate alarms based on fixed threshold exceedances. This approach produces high false alarm rates, contributes to alarm fatigue among clinical staff, and fails to detect subtle physiological deterioration patterns that precede critical events. Recent advances in multi-modal medical sensing, deep learning, and multi-agent systems offer new possibilities for continuous, accurate, and clinically actionable patient monitoring. This study proposes a Multi-Agent Intelligent Patient Monitoring System (MIPMS) that integrates multi-modality physiological data—continuous vital sign waveforms (ECG, PPG, respiratory), infrared thermal imaging for non-contact temperature mapping, and wearable inertial measurement unit (IMU) data for activity and posture assessment—through a deep learning-based fusion architecture. A multi-agent clinical decision support module decomposes the post-detection workflow into specialized tasks—vital sign pattern classification, thermal anomaly detection, alert prioritization, and clinical summary generation—each handled by a dedicated LLM-powered clinical agent. The multi-agent design enables context-aware reasoning that accounts for patient history, activity state, and clinical workflow, producing more accurate and actionable outputs than single-modality threshold-based alarm systems. Experiments conducted on three clinical monitoring datasets demonstrate that the proposed system achieves an average vital sign classification accuracy of 92.4%, a thermal anomaly detection accuracy of 89.1%, and an alert prioritization F1-score of 86.7%. The multi-agent alert system reduces the false alarm rate by 67% compared with threshold-based alarms while maintaining full sensitivity for genuine clinical deterioration events. The clinical summary generation agent achieves a semantic consistency rate of 84% with expert physician assessments. This study validates the effectiveness of combining multi-modal physiological data fusion with multi-agent clinical decision support for scalable, accurate, and interpretable intelligent patient monitoring.