Instruction-Guided Multimodal Medical AI for Imaging, Oncology, and Biomedical Decision Support

Abstract

Multimodal medical artificial intelligence is increasingly used to integrate imaging, molecular, clinical, and textual evidence for disease detection, prognosis prediction, and treatment decision support. This article proposes an instruction-guided multimodal medical AI framework that combines medical image analysis, oncology knowledge extraction, transcriptomic evidence, and clinical reasoning. The framework is designed to support tasks such as low-dose medical image interpretation, breast cancer DCE-MRI segmentation, melanoma detection, oncology risk assessment, and evidence-based biomedical decision support. Vision transformers and convolutional attention networks are used to extract visual features, while instruction-guided language models organize clinical and molecular evidence into interpretable summaries. Transcriptomic and phase-separation studies are incorporated as biomedical knowledge sources to improve the biological grounding of model explanations. The framework emphasizes reliability, traceability, and human-centered interpretation, particularly in high-stakes medical scenarios where algorithmic recommendations require clinical validation. By integrating multimodal imaging, molecular biology, and explainable AI, the article provides a cross-disciplinary approach to medical intelligence systems.